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WELCOME 


This  site  is  about  how  to  train  people,  especially  teams.  Part  of  it 
covers  figuring  out  what  to  train.  Part  of  it  covers  how  to  actually  do 
training. 

In  the  U.S.  alone,  about  $20  billion  is  spent  each  year  on  training. 
Some  of  this  is  a  worthwhile  investment:  Researchers  have  found  up 
to  a  30-to-l  difference  in  performance  between  high-training  and 
low-training  groups  in  some  fields.  On  the  other  hand,  some  of  it  is 
pretty  badly  spent.  With  both  increasingly  complex  tasks  and 
increasingly  fast  change,  the  need  for  good  training  is  clear,  and  it's 
reasonable  to  expect  the  importance  of  training  to  grow.  The 
question  we  address  is,  how  do  you  design  good  training? 
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GETTING  ORIENTED 


This  set  of  pages  provides  an  overview  of  training.  If  you're  already 

familiar  with  the  territory,  you  may  wish  to  skip  ahead. 

We'll  begin  with  the  four  main  issues  in  designing  training: 

1.  What  task  needs  to  be  done,  and  how  well?  This  is  often  easy  to 
say  in  a  general,  vague  way,  and  difficult  to  specify  in  detail.  We 
talk  about  this  in  cTraining  Basics>. 

2.  What  is  currently  getting  done?  Again,  saying  this  approximately 

.  is  a  lot  easier  than  saying  it  precisely.  This  is  covered  in  cTraining  - 

Basics>  and  in  more  detail  in  <How  To  Figure  Out  What  Your 
Team  Is  Really  Doing>. 

3.  What  changes  could  make  it  better?  Once  again,  easy  to  do 
vaguely  and  hard  to  do  with  accuracy  and  detail.  Addressed  in 
<How  To  Use  the  Training  Guide>  and  in  <The  Training  Guide> 
itself. 

4.  What  is  the  best  way  to  train  to  get  those  changes?  The  big 
question.  Which  combination  of  maybe  fifty  possible  training 
approaches  best  fit  the  budget,  the  training  goals,  the  resources, 
and  the  time-frame?  See  <The  Training  Guide>. 
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HOW  TO  DO  IT  WRONG 


Why  not  train  it  as  it  has  been  trained  in  the  past?  After  all,  people 
and  teams  have  achieved  great  performances  in  all  sorts  of  fields 
using  whatever  training  methods  have  been  traditional  in  those 
fields.  In  fact,  training  the  next  generation  as  you  were  trained  may 
be  a  perfectly  good  idea.  But  then  again,  it  might  not  be.  Previous 
results  may  have  been  achieved  inefficiently  and  despite  poorly 
designed  or  poorly  executed  training. 

First,  poorly  designed  training  wastes  time.  It's  a  rare  domain  where 
finding  the  shortest  training  route  to  the  target  performance  isn't 
highly  desirable.  There  are  all  sorts  of  ways  poor  training  can  waste 
time.  For  example,  for  years  the  Navy  taught  basic  principles  of 
electronics  to  their  trainees  before  introducing  them  to  their 
electronic  equipment  troubleshooting  tasks.  It  wasn't  until  someone 
actually  observed  good  troubleshooters  at  work  that  it  was 
discovered  that  the  bulk  of  the  basic  principle  material  was  never 
used.  The  best  electronic  troubleshooters  had  no  more  knowledge  of 
basic  principles  than  the  newest  training  graduate. 

Perhaps  even  worse,  it  wastes  motivation.  Level  of  motivation  is  the 
best  single  predictor  of  performance  and  improvement.  Suppose  you 
have  the  good  fortune  to  have  a  crop  of  trainees  who  are 
enthusiastic  and  highly  motivated.  They'll  pay  attention  to  their 
instruction,  and  they'll  work  hard  at  practicing.  But  if  people  are 
trained  in  the  wrong  thing,  or  trained  in  the  wrong  way,  they  won't 
get  better.  And  nothing  drains  enthusiasm  more  effectively  than  hard 
work  with  no  reward. 

The  training  investment  made  by  both  the  trainers  and  the  trainees 
should  pay  off  in  measurably  improved  performance.  The  way  you  do 
it  now  may  be  the  best  way  to  do  it.  If  there's  a  better  way,  though, 
it's  worth  finding  out. 
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UNDERSTANDING  WHAT  TO  TRAIN 

Good  training  depends  on  a  lot  of  factors.  Motivation  has  already 
been  mentioned,  and  it's  probably  impossible  to  exaggerate  it's 
importance.  Unmotivated  trainees  don't  learn  anywhere  nearly  as 
efficiently  as  motivated  ones.  Shared  goals,  trainee  confidence  in  real 
rewards  for  improved  performance,  and  for  teams,  a  real  sense  of 
membership  and  commitment,  all  contribute  to  setting  the  stage  for 
training.  Breakdowns  in  any  of  these  can  sabotage  a  training 
program. 

Four  other  factors  are  critical :  . .  . 7 

First,  explanation  isn't  the  same  as  performance.  Being  able  to 
explain  something  is  one  thing,  being  able  to  do  it  is  another. 

Otherwise,  considering  the  number  of  words  that  have  been  written, 
every  golfer  in  the  world  would  be  shooting  63's.  Because  it's  often 
easier  to  test  and  measure  verbal  descriptions  than  actual 
performance,  though,  the  ability  to  explain  a  task  is  too  frequently 
mistaken  for  success  in  training. 

There  are  some  work  areas— law,  for  example— where  the  ability  to 
argue  and  to  construct  convincing  explanations  is  task  performance. 

But  even  for  this  kind  of  task,  measuring  how  well  the  task  is  i. 

explained  (for  instance,  the  ability  to  recite  the  rules  of  argument  and 
rhetoric)  is  still  different  from  being  able  to  actually  do  the  task,  to 
come  up  with  strong  explanations  and  to  persuade  people  with  them. 

This  isn't  to  say  that  explanation  isn't  a  crucial  element  of  many  kinds 
of  training.  Telling  people  what  to  do  is  most  often  the  most  effective 
approach.  However,  mistaking  trainees'  ability  to  recite  back  the 
"right  way  to  do  it"  with  their  ability  to  actually  do  it  is  not 
satisfactory. 

The  second  issue  is  that  it  can  be  hard  to  figure  out  what  really 
works.  For  some  tasks,  the  relationship  between  cause  and  effect  is 
murky.  Sometimes,  this  is  simply  due  to  the  imperfection  of  current 
knowledge.  But  in  domains  where  effects  can  arise  from  many  causes 
and  causes  can  result  in  multiple  effects,  it  can  be  difficult  to  say  what 
actions  will  have  what  result(s).  If  work  is  done  in  a  dynamic 
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environment  -  an  environment  that  changes  over  time  -  there  may 
not  be  any  single,  simple  reason  why  a  task  is  performed  well  or 
badly.  There  may  be  a  constellation  of  factors,  and  if  the  task  has 
multiple  steps,  these  can  be  quite  difficult  to  tease  apart  because  it 
may  be  possible  in  subsequent  steps  to  make  up  for  lower 
performance  in  previous  steps.  In  this  kind  of  situation,  many  widely 
differing  explanations  of  performance  can  be  all  be  to  a  greater  or 
lesser  degree  true.  Even  outstanding  performance  can  be  neutralized 
by  unexpected  changes  in  the  environment. 

In  addition,  there  may  be  more  than  one  route  to  the  desired  task 
outcome.  If  Method  A  doesn't  work,  then  an  alert  individual  or  team 
may  try  Method  B  or  Method  C.  This  is  of  course  good,  but  it  also 
means  that  isolating  the  reasons  for  success  or  failure  can  be  very 
difficult  and  error-prone. 

Further,  all  sorts  of  work  areas  that  involve  intensive  human 
interactions— teaching,  sales,  law— have  a  significant  element  of 
chance,  of  histories  and  impulses  and  unpredictable  encounters  that 
can  have  a  big  effect  on  outcome  but  are  completely  beyond  the 
control  of  the  performer. 

Finally,  since  no  observer  comes  to  a  task  completely  free  of  prior 
beliefs  and  expectations,  different  observers  will  see,  remember,  and 
interpret  the  same  evidence  differently. 

If  you  as  a  training  designer  can't  tell  what  causes  good 
performance,  it  will  be  hard  to  design  a  training  plan.  If  you  believe 
you  know  what  causes  good  performance  and,  for  the  reasons  just 
stated,  you're  mistaken,  it  will  probably  be  harder. 

The  third  major  factor  is  the  transfer  from  training  to  real-world 
performance,  and  this  isn't  as  straightforward  as  it  might  seem. 
"Transfer"  measures  the  degree  to  which  training  actually  changes 
performance,  and  it's  surprisingly  complex.  People  are  highly  attuned 
to  situational  and  historical  variables,  many  of  which  can  be  very  hard 
to  duplicate  in  training.  Similarities  between  training  and  actual 
performance  may  appear  persuasive,  but  difficult-to-see  underlying 
dissimilarities  can  block  or  dilute  the  expected  transfer.  In  addition, 
"compartmentalization"  of  training  (an  a  priori  attitude  on  the  part  of 
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trainees  that  training  and  the  real  world  are  simply  not  related)  can 
isolate  expected  training  effects. 

The  final  factor  is  that  people  are  so  good  at  learning,  they  can  learn 
a  lot  in  training  that  you  never  intended  for  them  to  learn.  People  will 
learn  all  sorts  of  things  about  the  trainer,  the  organizational  context 
in  which  the  training  takes  place,  the  actual  goals  of  the  training 
(when  these  are  different  from  the  stated  goals),  how  much  is 
actually  expected  of  them,  and  a  host  of  other  things  that  may  have 
little  to  do  with  the  ostensible  goals  of  the  training. 

Assuming  all  of  these  factors  can  be  addressed  to  a  greater  or  lesser 
degree,  the  next  step  is  to  look  at  the  issues  of  training  design. 
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TRAINING  DESIGN 

Because  people  are  great  at  learning,  there  are  almost  always  many 
ways  to  achieve  training  goals.  For  the  same  reason,  there  are 
almost  always  many  reasons  why  goals  are  achieved.  (It's  common 
enough  that  training  goals  are  achieved  despite,  rather  than  because 
of,  a  training  program.)  If  you  mistake  the  reason(s)  a  training 
approach  worked  for  a  particular  task,  which  may  be  easy  to  do,  then 
1  you're  more  likely  to  combine  an  inappropriate  training  design  with 
high  expectations  on  the  next  task. 

The  "Hawthorne  effect"  further  complicates  training  design,  because 
it  makes  it  harder  to  tell  whether  or  why  a  change  in  training 
1  approach  works.  In  the  1950's,  it  was  noticed  that  whenever  a  new 
educational  reform  was  introduced,  a  test  classroom  demonstrating 
the  new  approach  would  show  a  significant,  even  striking 
improvement  in  learning.  The  approach  would  be  widely  adopted,  but 
the  results  would  be  disappointing.  Following  the  disappointment,  a 
new  approach  would  be  developed,  pretty  much  the  opposite  of  the 
previous  one.  A  test  classroom  would  again  show  a  big  improvement 
in  learning.  So  now,  the  new  approach  would  be  widely  adopted.  But 
it  would  turn  out  that  it,  too,  didn't  really  improve  things.  Why  did  the 
test  classrooms  do  so  well,  while  the  wide  adoption  didn't?  The 
Hawthorne  effect:  The  improvement  had  very  little  to  do  with  the 
differences  in  teaching  approach,  but  everything  to  do  with  the 
enthusiasm  and  special  attention  lavished  on  the  test  classrooms. 

Training  design  may  also  have  to  address  not  only  how  to  do  task 
but  how  to  (continuously)  learn  how  to  do  the  task  better.  If  the  job 
being  trained  is  something  that  is  constantly  changing  or  evolving, 
then  training  has  to  address  dynamic  adaptation:  Not  just  how  you 
do  the  work,  but  how  you  change  how  you  do  it  to  adapt  as  the  thing 
you're  doing  changes.  In  any  field  where  there  is  a  significant 
technology  element,  this  is  likely  to  be  an  issue.  And  depending  on 
the  specifics  of  the  job  being  trained,  this  can  be  an  especially 
sensitive  area  for  interaction  of  training  with  long-term  motivation. 

Once  you've  designed  a  training  program,  you  need  to  test  it.  In 
some  fields— police  and  firefighting,  medicine,  military  affairs— 
performance  data  can  be  hard  to  get.  In  some  cases,  even  if  it  exists, 
it  is  considered  sensitive  enough  that  it  may  not  be  available  for 
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inspection.  Getting  a  good  picture  of  the  effect  to  training  on 
performance  can  require  integration  of  partial  evidence  from  a  large 
number  of  sources,  each  of  which  may  vary  in  the  degree  to  which 
they  are  similar  to  the  actual  conditions  for  which  you  want  to  train. 
This  is  easily  connected  to  a  a  temptation,  often  entirely  unconscious, 
to  "test  what  you've  trained,"  just  because  testing  for  the  real  point 
of  the  training,  namely,  real-world  performance,  is  so  hard. 
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DOING  IT  BETTER 

We've  just  described  some  reasons  why  training  is  hard.  But  there's 
a  bright  side,  too.  Even  though  to  develop  a  good  training  program,  a 
lot  of  decisions  with  uncertain  answers  must  be  made,  the 
uncertainty  of  the  answers  can  be  greatly  reduced  with  improved 
methods  for  understanding  tasks  and  selecting  training  methods: 

•  Task  analysis  techniques  have  multiplied  and  been  refined  into 
powerful  tools  for  analysis  of  individual  and  team  behavior. 

•  Cognitive  task  analyses  can  lead  to  better  understanding  of  the 
mental  roots  of  performance,  performance  breakdown  and  stress 
points,  and  better  design  of  appropriate  training  materials. 

•  Metacognitive  analyses— improved  understanding  of  how  we 
control  our  own  thinking,  how  and  for  how  long  we  focus  our 
attention,  when  we  know  we  don't  know  something,  and  so  on— 
and  explicit  training  of  domain-specific  metacognitive  techniques 
have  been  shown  to  lead  to  significant  improvements  in  task 
performance. 

•  Organizational,  sociological,  and  training  researchers  have 
developed  insights  into  the  nature  and  operation  of  teams.  This 
has  laid  a  foundation  for  group  cognitive  task  analyses  to  provide 
tools  and  techniques  for  better  understanding  of,  and  therefore 
training  of,  group  behaviors  and  performance. 

There's  a  lot  of  hard  work  involved  in  developing  a  training  program 
and  seeing  it  through  implementation  and  testing.  But  insights  about 
learning  and  human  capabilities  (and  frailties)  that  have  emerged 
from  experimental  and  evolutionary  psychology,  cognitive  science, 
artificial  intelligence,  and  the  decision  sciences  have  all  contributed  to 
better  tools  for  understanding  human  performance.  At  the  same  time, 
improvements  in  technology  have  made  some  kinds  of  training 
approaches,  especially  computer-based  simulation  training,  both 
possible  and  affordable. 

So  -  how  to  do  it?  As  we  proceed,  we'll  propose  a  systematic  method 
for  determining,  for  a  particular  task  or  competence,  what  is  the  most 
effective  way  to  train. 
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BASICS  OF  TRAINING  &  TRAINING  DESIGN 


In  this  set  of  pages  we  go  over  the  basics.  We  start  by  talking  about 
learning  -  its  different  forms,  as  seen  by  the  learner.  Then  we  define 
just  what  we  mean  by  "training"  and  "training  design."  We  describe 
the  eight  characteristics  of  teams  that  are  important  for  training.  This 
section  concludes  with  a  review  of  the  nine  steps  in  training  design. 

<Learning> 

<How  learning  works> 

<Expertise> 

<What  training  is> 

<Training  design> 

<Teams> 

<The  ten-step  approach> 
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LEARNING 


Learning  is  goal-driven  persistent  adaptation  to  a  perceived 
environment. 

OK,  what  does  that  mean?  First,  learning  is  powered  by  the  impetus 
to  achieve  some  goal.  The  impetus  doesn't  have  to  be  conscious  - 
infants  don't  make  well-considered  decisions  about  whether  to  learn 
to  talk.  But  even  the  earliest  learning,  the  most  strongly  biologically 
programmed  infant  development,  is  altered  by  the  strength  of  the 
goals.  Babies  who  have  colorful  mobiles  to  reach  for  learn 
coordination  more  quickly  than  those  who  don't. 

Second,  it  only  counts  as  learning  if  it  lasts.  If  you  forget  the  lesson 
the  moment  you  walk  out  the  door,  it  doesn't  count  as  learning.  This 
doesn't  mean  all  learning  has  to  be  conscious,  of  course  -  simply  that 
it  has  to  last  long  enough  to  be  of  service  to  the  goal(s). 

Third,  learning  usually  means  a  change  that  makes  you  better  at 
achieving  your  goal.  This  is  a  little  tricky,  because  some  goals  are 
members  of  "goal  classes."  You  might  not  learn  how  to  solve  the 
current  problem,  but  something  that  makes  you  better  at  solving  that 
kind  of  problem.  It's  also  tricky  because  achieving  some  goals 
requires  going  backwards  before  you  can  go  forward.  As  a  simple 
example,  sometimes  learning  more  options  can  make  it  harder  to 
decide  what  to  do. 

Finally,  learning  never  takes  place  in  a  vacuum  -  it  always  happens  in 
some  environment.  And  which  elements  of  that  environment  are 
perceived  in  what  ways  has  a  big  effect  on  what  is  learned.  Since  a 
big  part  of  training  design  is  the  design  of  environments,  this  is 
important. 

Learning  is  an  active,  constructive  process,  in  which  the  learner  is  the 
principal  actor.  The  learner  actively  builds,  tests,  repairs,  and  refines 
his  or  her  understanding,  skills,  and  abilities.  Learning  is  action. 
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HOW  LEARNING  WORKS 


Roger  Schank  and  Chip  Cleary  describe  learning  as  a  three-stage 
process,  the  "learning  waterfall." 


Adopt  a  goal] 


Learning  by  doing 
Incidental  learning 


Generate  a 


guestL 


Learning  by  musing 


(reflection) 


Develop  an  answi 


Case-based  teacl 
Learning  by  exp! 


They  observe  that  people  spontaneously  learn  about  things  that  are 
interesting  to  them,  that  is,  that  pertain  to  their  goals.  We  move  from 
specific  experiences  to  more  general  understanding,  what  Schank 
and  Cleary  call  "rich  generalization,"  so-called  because  since  real 
experience  always  includes  exceptions  and  borderline  cases,  our 
spontaneous  generalizations  are  fluid  and  flexible,  in  contrast  to  dry, 
memorized  textbook  rules. 

We  pursue  our  goals,  learning  as  we  do  things  and  as  we  encounter 
things  along  the  way.  We  consider  what  we  want  to  do  next 
(generate  a  question),  and  we  learn  in  the  process  of  figuring  out 
what  to  ask,  who  to  ask,  how  to  ask,  what  perspectives  are  possible, 
what  can  be  articulated,  how  to  move  forward,  what  isn't  known, and 
so  on.  To  develop  an  answer,  we  think  about  similar  situations,  recall, 
compare,  and  evaluate  stories  and  things  we've  read  or  been  told; 
we  try  things  out,  perform  experiments,  and  we  practice  what  we've 
learned. 

Schank  and  Cleary  claim  that  the  more  teaching  and  training  cleave 
to  the  waterfall,  the  less  work  and  the  more  benefit  there  will  be. 
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EXPERTISE 


People  have  always  admired  experts,  but  as  an  area  of  empirical 
research,  it  has  a  short  history.  DeGroot's  experiments  on  chess 
players  (1946)  were  the  first  systematic  studies  of  experts. 

The  current  research  consensus  on  expertise  is  that  it  is  a  function  of 
both  knowledge  and,  probably  more  importantly,  knowledge 
organization.  For  example,  physics  graduates  will  search  through 
their  knowledge  to  find  the  right  problem  solving  techniques,  but 
experienced  practicing  physicists  will  simply  "see"  the  appropriate 
approach  as  part  of  understanding  the  problem.  The  knowledge  is 
organized  in  such  a  way  that  they  can  apply  it  quickly  and  fluidly. 

It  typically  takes  no  less  than  ten  years  to  develop  expertise  -  and  a 
decade  of  practice  appears  to  be  necessary,  but  not  sufficient.  Just 
doing  the  same  thing  for  ten  years  does  not  an  expert  make.  How 
you  do  it  counts. 

The  key  to  development  of  expertise  seems  to  be  "deliberate 
practice."  Deliberate  practice  is  practice  that  meets  four  extra  criteria: 

•  The  task  is  appropriate.  That  is,  the  thing  being  practiced  is  a 
useful  thing  to  practice  in  order  to  increase  the  learner's 
capability.  (This  should  go  without  saying,  but  unfortunately,  it 
does  not  always  hold  true.) 

•  Appropriate  level  of  difficulty.  Hard  enough  to  be  challenging,  not 
so  hard  as  to  engender  bad  habits  or  discourage  further  practice. 

•  Informative  feedback.  The  learner  can  tell  or  be  told  the  effects  of 
his  or  her  actions. 

•  Opportunity  for  repetition  and  correction  of  errors.  Things  that 
need  to  be  corrected  or  improved  can  be  worked  on  directly. 

These  may  all  seem  obvious,  but  for  each  one,  counterexamples  are 
easy  to  find.  At  any  given  time  in  a  crowded  elementary  school 
classroom,  the  task  is  unlikely  to  be  appropriate  for  a  large  number  of 
students.  A  Marine  commander  doesn't  get  much  chance  for 
repetition  and  correction  of  errors. 

Training  to  develop  expertise  is  a  long-term  project. 
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WHAT  TRAINING  IS 


Training  is  any  activity  that  produces  a  persistent  change  in 
behavior  or  competence,  in  a  person  or  group,  with  respect  to 
a  task  or  family  of  tasks. 

The  training  goal  should  always  be: 

Do  it  better  in  the  world. 

Why  is  it  necessary  to  say  this?  Because  there  are  plenty  of 
places  where  it  is  completely  ignored.  How  many  schools 
reward  students  for  little  other  than  being  good  at  going  to 
school? 

For  some  kinds  of  work,  an  additional  training  goal  may  be  to 
improve  the  trainees'  self-directed  ability  to  to  learn  on  the  job, 
so  that  the'  competence  continues  to  increase  long  after  formal 
training  is  completed. 
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TRAINING  DESIGN 


Training  design  is  the  selection  or  creation  of  environment(s),  events, 
and  sequences  for  an  individual  or  group  so  that  a  desired  (actual 
task)  performance  capability  arises  and  persists. 

In  other  words,  what  do  you  want  them  to  see,  hear,  feel,  and  think, 
in  order  to  be  able  to  do  what  they're  supposed  to  do? 

The  environment  and  its  content  are  conditioned  by  social,  ethical, 
political,  and  aesthetic  values.  You  may  have  to  consider  all  of  these 
in  training  design. 

"Training  transfer"  is  what  shows  up  in  the  real  world  as  a  result  of 
training.  The  basic  rule  of  transfer  from  training  to  practice:  The  more 
similar  the  environment(s),  events,  and  sequences  of  training  are  to 
those  of  the  real  task,  the  more  effective  the  transfer. 
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THE  NINE-STEP  APPROACH  TO  TRAINING  DESIGN 


Gagne,  Briggs,  and  Wager  (1992)  propose  a  nine-step  instructional 
design  sequence. 

1.  Define  the  instructional  goals. 

Instructional  goals  address  the  gaps  between  the  desirable  state  of 
affairs  and  the  current  or  observed  state  of  affairs.  In  other  words, 
what  can't  they  do  that  you  want  them  to  be  able  to  do?  New  ideas 
or  new  technology  can  also  call  for  creation  of  instructional  goals. 

2.  Conduct  instructional  analysis 

"Instructional  analysis"  is  the  process  of  describing  the  task(s)  to  be 
trained.  Representative  kinds  of  instructional  analyses  are  task 
analysis,  cognitive  task  analysis,  and  learning  analysis.  The  various 
task  and  cognitive  task  analyses  are  covered  later  cINSERT  LINKS 
HERE>.  Learning  analysis  discriminates  between  terminal  and 
enabling  objectives,  that  is,  the  final  result  and  what  they  need  to 
know  or  be  able  to  do  in  order  to  achieve  the  final  result.  It  also 
addresses  the  sequence  of  enabling  objectives,  that  is,  what  to  learn 
first.  (Why  educators  need  to  say  "sequencing  of  enabling  objectives" 
instead  of  "what  to  learn  first"  is  beyond  me.) 

3.  Identify  entry  behaviors  and  learner  characteristics 
Determine  which  required  enabling  skills  the  learners  already  have, 
and  note  any  other  characteristics  of  the  learners  germane  to 
developing  instruction.  That  is,  figure  out  who  you  are  training. 

4.  Identify  performance  objectives 

Translate  needs  and  goals  into  specific,  detailed  objectives.  Detail  is 
necessary  both  to  development  of  training  materials  and  to  creation 
of  measurement  tools  (colloquially  called  "tests")  to  assess  student 
progress  and  determine  the  degree  of  success  in  achieving  the 
objective(s).  A  good  learning  objective  has: 

•  A  statement  of  the  behavior  (action)  or  understanding  the 
trainee  must  show,  that  is,  the  desired  result  of  training.  This  can 
be  either  positive  or  negative.  For  situation  A,  the  goal  may  be  to 
perform  X,  or  it  may  be  to  avoid  or  refrain  from  performing  X. 

•  Specification  of  the  conditions  under  which  the  behavior  must 
take  place. 
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A  statement  of  the  standards  for  satisfactory  performance. 


Some  kinds  of  training  goals  are  easier  to  characterize  via  the 
behavior/  conditions/standards  approach  than  others.  For  example, 
"teamwork"  is  harder  to  specify  and  measure  than  "operating  a 
radio."  Getting  the  statement  of  a  training  objective  right  is  critical. 
Conditions  and  standards  are  frequently  inadequately  specified  in 
training  system  development. 

Gagne  et  al  identify  five  classes  of  outcome. 

1  Intellectual  skills.  Learning  how  to  do  something  of  an  intellectual 
sort.  The  ability  to  interact  with  the  environment  (broadly 
construed)  in  terms  of  symbols  and  concepts,  and/or  the  learning 
of  procedural  knowledge. 

2  Cognitive  strategies.  Skills  for  governing  thinking.  This  includes 
both  operational  and  predictive  knowledge.  For  example,  having 
appropriate  expectations,  and  knowing  how  much  certainty  to 
invest  in  them,  is  a  critical  skill  of  competent  commanders. 

3  Verbal  information.  Declarative  knowledge,  and  the  ability  to 
integrate  it  with  other  knowledge  and  to  recall  it  appropriately. 

4  Motor  skills.  Physical  skills,  and  (to  the  extent  not  covered  by 
other  categories)  knowledge  of  how  and  when  to  apply  them. 

5  Attitudes.  A  persisting  state  within  the  affective  domain  that 
consistently  influences  or  modifies  an  individual's  choice  of 
actions.  Attitudes  are  immensely  important  in  some  work  -  police, 
the  military,  athletics. 

Within  intellectual  skills,  there  are  five  subcategories:  problem¬ 
solving,  rules,  defined  concepts,  concrete  concepts,  and 
discrimination.  In  reverse  order: 

•  Discrimination  involves  distinguishing  features  of  objects  or 
situations  as  similar  or  different,  e.g.,  a  red  Ford  Taurus  is  the 
same  as  a  blue  Ford  Taurus  except  for  the  color. 

•  Concrete  concepts  involve  identifying  a  class  of  object 
characteristics,  objects,  or  events,  e.g.,  "that's  a  Ford  Taurus". 

•  Definitional  concepts  involve  classification  of  an  object  or  event  in 
accordance  with  a  definition,  e.g.,  a  Ford  Taurus  is  a  domestic 
car. 

•  Rules  require  knowing  rule  content,  conditions,  and  in  most  cases 
exceptions,  along  with  the  competence  to  apply  the  rule 
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correctly.  For  example: 

Water  expands  when  it  freezes  => 

Don't  freeze  water  in  an  inelastic  container  => 

Don't  fill  your  Taurus'  cooling  system  with  water  if  the 
temperature  is  likely  to  go  below  0°  C. 

•  Problem-solving  involves  generating  a  solution  to  a  novel 

problem,  e.g.,  predicting  the  sale  of  right-hand  drive  Tauruses  in 
Japan. 

5.  Develop  criterion-referenced  test  items 

Criterion-referenced  test  items  determine  whether  the  students  have 
acquired  the  desired  skill,  rather  than  merely  showing  they  remember 
the  instruction.  These  are  necessary  for  training  evaluation.  They  can 
also  be  used  to  place  students  within  a  training  program,  by 
determining  which  of  the  enabling  and  terminal  objectives  they  have 
already  mastered. 

6.  Develop  instructional  strategy  plan 

Instructional  strategy  is  a  plan  for  assisting  learners  with  their  study 
efforts  for  each  performance  objective.  It  is  a  mapping  of  instructional 
activities  to  accomplishment  of  objectives,  as  well  as  diagnosis  and 
prescription  of  the  most  apt  instructional  approach  for  the  objective. 

7.  Develop  instructional  materials 

The  selection  and  development  of  instructional  materials. 

8.  Conduct  formative  evaluation 

"Formative"  means  that  the  course  is  being  revised  and  tuned 
through  this  evaluation.  A  three-step  approach  is  advocated  by  Dick 
and  Carey  (1990).  First,  try  the  materials  one-on-one.  Then,  try  the 
materials  on  a  small  group  of  students.  Then,  perform  a  field  trial  in 
which  the  instruction  is  revised  based  on  the  whole  class.  (This  is  of 
course  subject  to  the  kind  of  objectives  and  material  involved.) 

9.  Conduct  summative  evaluation 

Evaluate  the  course  or  training  program  after  it  has  been 
implemented  and  put  in  place. 

There  is  a  lot  more  than  these  nine  steps  to  instructional  design.  In 
actually  developing  a  training  course,  resources,  the  environment, 
other  constraints,  teacher  preparation,  installation,  and  diffusion  all 
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have  to  be  considered. 
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TEAMS 


Teamwork  is  different  from  "taskwork."  Taskwork  is  what  individuals 
do  by  themselves.  Sometimes  "teams"  are  just  groups  of  people  who 
do  their  individual  tasks  side-by-side.  An  assembly  line  is  an  example 
of  this  kind  of  taskwork.  Teamwork  happens  when  team  members 
need  to  interact  to  achieve  common  goals,  and  their  interaction 
includes  adapting  to  changing  circumstances. 

There  are  eight  elements  of  teamwork: 

1.  Shared,  usually  conscious  goal(s) 

2.  Mutual  monitoring 

3.  Communication 

4.  Feedback 

5.  Reciprocal  backup 

6.  Membership  awareness 

7.  Interdependence 

8.  Shared  skills 

Sharing  goals,  usually  consciously,  is  pretty  much  the  basic  thing  that 
distinguishes  a  team  from  a  group  of  people  who  just  like  to  hang  out 
together.  Seriously,  a  major  mark  of  team  coherence  is  the  degree  to 
which  team  members  share  goals,  and  even  moreso,  the  degree  to 
which  they  are  willing  to  put  aside  their  personal  desires  in  favor  of 
team  success. 

More  on... 

<Mutual  monitoring> 

<Feedback> 

<Communication> 

< Reciprocal  backup> 

< Membership  awareness> 

<Interdependence> 

<Shared  skills> 

<More  aspects  of  team  tasks> 
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TEAMWORK:  MONITORING 

Monitoring  includes  observing  teammates  behavior,  checking  that 
procedures  or  actions  are  correct  and  timely,  that  teammates  are  not  in 
need  of  assistance,  that  appropriate  progress  is  being  made  toward 
the  goal  of  the  team. 

Issues  affecting  monitoring  include:  Shared  mental  models  of  the  task, 
including  mental  models  of  other  team  members'  expected  behavior, 
and  mental  models  of  other  team  members'  beliefs  and  attitudes; 
mutual  trust;  ease  of  monitoring  (physical  proximity,  clarity  of  cues  as 
to  performance  status,  obviousness  of  engagement);  and,  the  pace  of 
change  during  task  performance. 

If  team  members  are  not  close  together  during  task  performance,  then 
shared  mental  models  of  task  performance  are  important  to  maintain 
actual  and  perceived  synchronization.  Even  if  an  individual  is  being 
trained  for  a  specialty,  he  or  she  should  receive  training  in  other 
relevant  task  activities. 

If  diverse  backgrounds  are  common,  training  in  both  expected 
standards  of  professionalism  and  specific  culture-sensitive  modes  of 
communication  should  both  be  considered. 

If  the  pace  of  change  during  task  performance  is  very  rapid,  or  has  the 
potential  to  be  rapid,  then  training  in  modeling  task  performance, 
including  critical  incident  training,  should  be  combined  with  active 
practice  at  anticipation  of  possible  situation  development  and 
teammate  performance  trajectories. 
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TEAMWORK:  FEEDBACK 

Feedback  is  a  follow-up  activity  to  monitoring.  Giving  and  receiving 
feedback  is  a  critical  technique  for  taking  advantage  of  strengths  and 
compensating  for  weaknesses  in  a  team,  and  for  group  improvement 
and  optimization. 

Issues  influencing  feedback  include  whether  it  is  direct  or  mediated  (for 
example,  through  a  mutual  superior);  the  roles  of  status,  tenure,  and 
rank  in  affecting  feedback  flow;  whether  feedback  is  explicitly  or 
implicitly  communicated  (implicit  communication  might  be  through 
attitudes,  degrees  of  cooperation,  physical  postures,  etc.);  whether 
there  are  "post-event"  reviews;  and  whether  there  is  any  potential  for 
reprisal  for  unwanted  feedback. 
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TEAMWORK:  COMMUNICATION 


Communication  is  critical  not  only  in  feedback,  but  is  the  way  many 
tasks  are  coordinated  and  executed.  Communication  issues  include 
whether  there  is  an  open  channel,  a  constrained  ("only  when  asked") 
channel,  a  clogged  channel,  or  a  closed  channel;  whether 
communication  is  task  oriented  only  or  includes  a  wider  range  of  topics; 
whether  communication  can  occur  at  any  time,  or  only  at  designated 
times;  and  whether  there  are  special  methods  to  "close  the  loop" 
(confirm  communication  correctness). 

If  a  particular  kind  of  channel  is  organizationally  sanctioned,  trainees 
should  know  it  and  how  to  behave  appropriately.  If  the  nature  of  the 
channel  is  dependent  on  individual  leaders'  or  teams'  styles,  training 
should  prepare  trainees  to  adapt  to  whatever  style  they  encounter, 
with  professionalism  "from  below." 

Some  tasks  have  specific  task-oriented  communications,  like  the 
copilot's  role  in  providing  information  to  the  pilot  when  landing  a  plane. 
Such  communications  are  often  somewhat  formalized;  appropriate 
forms  should  be  trained  and  rehearsed.  In  some  tasks,  the  task 
pressure  is  great  enough  to  perturb  normal  communication;  if  this  is 
known  to  happen  sometimes,  immersion  in  (simulated)  task  conditions 
is  appropriate. 

If  communication  is  to  happen  at  designated  times,  or  if  time  for 
communication  is  limited,  then  thorough  familiarity  with  task-specific 
jargon  is  necessary.  Classroom  introduction  should  be  followed  by 
rehearsal,  practice,  and  feedback. 

If  closing  the  loop  is  part  of  the  job,  "false  model  training"  might  be 
used  to  augment  normal  training,  in  order  to  prevent  trainee  guessing 
and  covering  of  misunderstandings. 
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TEAMWORK:  RECIPROCAL  BACKUP 


Backing  each  other  up  is  the  hallmark  of  teamwork.  Feeling  welcome  to 
jump  in  if  the  need  is  perceived,  and  accepting  others' jumping  in 
without  fearing  being  perceived  as  weak,  indicate  strong  reciprocal 
backup. 

Some  reciprocal  backup  issues  include  the  degree  of  task  specialization 
versus  the  pool  of  common  skills;  whether  there  is  physical  task  trading 
and/or  sharing;  whether  there  is  emotional  or  attitudinal  task  trading 
or  sharing  (as  in,  for  example,  medicine);  whether  there  is  intellectual 
task  trading  and/or  sharing;  the  role  of  anticipation  in  backing  up 
appropriately;  and,  the  role  of  "anticipatory  compensation,"  that  is,  of 
doing  something  so  backup  is  less  likely  to  be  needed. 


TEAMWORK:  MEMBERSHIP  AWARENESS 


In  general,  team  members  must  be  aware  of  themselves  as  part  of  a 
team,  see  themselves  as  team  members,  and  be  willing  to  put  team 
success  above  individual  goals.  Some  issues  that  influence  this  include 
explicit  markers  of  team  membership  (uniforms,  physical  separation, 
special  tokens,  special  treatment);  a  sense  of  being  part  of  team;  pride 
or  other  intangible  elements  of  team  membership  (like  a  sense  of  being 
recognized  as  part  of  team);  explicit  expectations  for  team  obligations; 
explicit  expectations  for  team  benefits;  and,  the  notion  that  the  team  is 
more  important  than  the  individual,  that  the  team  member  will  sacrifice 
individual  goals  to  the  team  goal. 

Putting  the  team  above  individual  desires  is  probably  the  single  most 
important  aspect  of  teaming.  It  is  hard  to  achieve— in  athletics,  coaches 
are  always  emphasizing  team  (Gruff  voice:  "Men,  there's  no  T  in 
team.")  External  focus  during  training  on  the  importance  of  shared 
goals  can  help;  minimizing  any  "star  system"  can  also  help. 
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TEAMWORK:  INTERDEPENDENCE 

Teamwork  is  defined  as  involving  interaction.  Part  of  that  is  the  attitude 
of  interdependence:  willingness  to  accept  interdependence,  confidence 
in  teammates,  assumption  of  responsibility,  perception  of  reliability. 

Some  interdependence  issues  are  the  degree  to  which  each  task  is 
structurally  intertwined  with  the  others;  the  degree  to  which  attitudinal 
and  motivational  factors  require  perceived  support;  the  degree  to 
which  the  nature  of  the  task  encourages/requires  interdependence; 
and,  the  degree  to  which  the  task  culture  supports  interdependence 
(for  example,  firefighters  living  together  when  on  duty). 

If  tasks  are  structurally  intertwined,  the  on  the  job  interdependence 
will  be  functionally  required.  Training  in  standards  of  professionalism 
can  help  this  go  smoothly. 

If  the  nature  of  the  task  requires  interdependence  (firefighting  and 
police  work  share  the  characteristic  that  one's  life  may. depend  on  the 
alertness  and  competence  of  coworkers),  the  perceived  rigor  of  training 
and  ongoing  OJT  support  development  of  appropriate  attitudes. 

If  the  task  culture  does  not  explicitly  support  development  and 
maintenance  of  attitudinal  interdependence,  then  training  should 
explicitly  emphasize  interdependence,  e.g.,  by  simulating  task 
performance  with  dependent  support  withdrawn.  In  addition, 
appropriate  expectations  should  be  trained. 
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TEAMWORK:  SHARED  SKILLS 

It  is  often  the  case  that  different  circumstances  call  for  different  styles 
of  teamwork.  In  one  situation,  a  clear  and  commanding  leader  may  be 
needed;  in  another,  a  more  organic  group  adaption  is  appropriate;  and 
in  a  third  a  democratic  approach  might  be  best.  But,  different  tasks  call 
for  different  degrees  of  stylistic  flexibility.  There  are  two  issues:  Degree 
of  flexibility  required  of  team  with  respect  to  style,  and  degree  of 
autonomy  of  subteams  once  subtask  assignments  are  made. 

If  a  great  degree  of  style  flexibility  is  needed,  then  specific  role-playing 
exercises  are  potentially  useful.  The  culture  of  the  organization 
(including  especially  the  perception  that  leaders  are  selected  fairly 
according  to  appropriate  standards)  is  also  important. 

If  subteam  autonomy  is  important,  role  playing  within  subteams,  and 
clear  instruction  on  how  authority  is  managed  are  both  suggested. 
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TEAMWORK:  MEMBERSHIP  STABILITY 


Teams  change  over  time.  This  change  can  be  the  result  of  turnover  in 
members,  or  in  stable  teams,  mutual  adaption,  trust,  and  team 
efficiency. 

The  issues:  How  membership-stable  are  teams?  And,  how  important  is 
membership  stability,  i.e.,  how  much  tacit  team  adaption  to  incident 
circumstances  is  shaped  by  team  members'  knowledge  of  one 
another's  strengths,  weaknesses,  and  preferences? 

If  teams  are  stable,  and  the  rate  of  introduction  of  new  members  is 
gradual,  then  the  overall  degree  of  attitudinal  training  may  need  to  be 
less,  and  more  training  can  safely  be  on  the  job  (OJT). 

If  stability  is  important,  then  training  in  standards  of  professionalism  is 
useful. 
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MORE  ASPECTS  OF  TEAM  TASKS 


To  design  training,  of  course  you  need  to  know  your  subject.  It  is  also 
useful,  though,  to  understand  your  subject  in  a  more  abstract  way. 
The  advantage  of  this  is  in  allowing  identification  of  tasks  with  similar 
characteristics.  If  you  can  do  that,  then  the  pool  of  training  practices 
from  which  you  can  draw  is  just  that  much  richer. 

What  do  we  mean,  though,  by  an  abstract  understanding  of  a 
subject?  We've  just  listed  eight  abstract  characteristics  of  teamwork, 
on  each  of  which  any  particular  job  or  task  could  be  ranked.  Here,  in 
more  compact  form,  are  ten  more.  Above,  we  looked  at  teams 
directly.  Now,  we  consider  some  of  the  elements  of  the  tasks  for 
which  teams  are  formed  Does  your  team... 

1.  Perform  diverse  tasks,  which  vary  in  both  criticality  and  urgency. 
"Criticality"  is  how  important  a  thing  is.  "Urgency"  is  how 
important  timing  is  to  a  particular  activity.  People  sometimes 
confuse  urgency  with  criticality,  thinking  that  anything  that  has  to 
be  done  Real  Soon  must  be  important  for  that  reason  alone. 

2.  Coordinate  multiple  interacting  tasks  to  meet  overall  objectives. 

3.  Adapt  to  unanticipated  demands  and  opportunities  for  action 
generated  by  dynamic  external  entities.  Fire,  police,  military,  and 
athletic  teams  all  do  this.  Company  teams  do  it  too,  over  longer 
time-frames. 

4.  Satisfy  real-time  constraints.  That  is,  work  with  real  deadlines,  as 
in  medicine. 

5.  Operate  within  resource  constraints.  Who  doesn't?  But  for  some 
teams  and  tasks,  resources  are  a  bigger  problem  than  for  others. 

6.  Operate  within  human  motivational,  perceptual,  reasoning,  and 
executive  constraints.  This  is  a  bigger  issue  in  the  military  than  in 
most  other  areas,  those  some  brands  of  athletics  also  encounter 
it. 

7.  Operate  within  intrinsically  uncertain  information  and  knowledge 
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environments.  Again,  no  task  enjoys  perfection,  but  it's  more  of 
an  issue  for  some  kinds  of  work  than  for  others. 

8.  Dynamically  revise  and  modify  goals. 

9.  Deal  with  "identifiability"  issues,  that  is,  uncertain  association  of 
effect  with  cause,  in  an  environment  where  events  can  have 
multiple  causes  and  causes  can  trigger  multiple  events,  and 
where  causes  and  events  can  be  quite  distant  from  one  another 

10.  Perform  under  stress. 

It  probably  seems  complicated  to  try  to  keep  track  of  all  of  these 
factors.  It  does  to  us.  In  a  later  section,  we'll  provide  an  automated 
way  to  assess  your  team  and  its  work. 
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TEAMWORK  RESEARCH  FINDINGS 


Our  research  goal  was  to  look  at  the  interrelations  between  tasks 
and  teamwork.  Do  dangerous  tasks  have  higher  levels  of  mutual 
monitoring?  Do  tasks  in  which  errors  are  potentially  very  serious 
have  higher  levels  of  reciprocal  backup? 

In  this  section  we  describe  our  research  on  teamwork.  After  a  brief 
recap  of  the  eight  elements  of  teamwork,  we  discuss  both  problems 
in  the  study  of  teamwork  and  the  reasons  to  expect  continued 
progress. 

We'll  then  describe  our  data  gathering  and  analysis  in  detail.  The 
section  ends  with  our  assessment  of  the  results. 
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ELEMENTS  OF  TEAMWORK:  MICROREVIEW 


We've  reported  the  common  distinction  between  taskwork  and 
teamwork:  Taskwork  is  side-by-side,  either  in  parallel  or  serially, 
while  teamwork  is  interacting  and  mutually  adaptive,  with 
perceived  common  circumstances  and  shared,  valued  goal(s). 

The  eight  elements  of  teamwork  are: 

Mutual  monitoring 
Feedback 
Communication 
Reciprocal  backup 
Membership  awareness 
Interdependence 
Shared  skills 

Stability  and  development 
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PROBLEMS  OF  CLASSIFICATION,  REASON  FOR  HOPE 

The  ideal  thing  would  be  to  have  a  taxonomy  of  task  types  with 
associated  best-possible  training  approaches.  This,  however,  is 
harder  than  it  seems. 

There  are  many  taxonomies  of  human  work  and  performance.  The 
first  problem  with  construction  of  a  taxonomy  is  deciding  on  what 
basis  to  classify.  Do  you  classify  by  aspects?  If  so,  which  aspects,  at 
what  level  of  abstraction?  Or  by  function?  Or  by  generic  task 
category?  Or  by  subject  matter?  Or  according  to  a  particular 
epistemology?  Or  some  hybrid  combination  of  any  or  all  of  these? 

Existing  taxonomies  have  to  some  extent  been  overtaken  by  events. 
As  our  understanding  of  cognition  grows,  many  of  the  assumptions 
upon  which  taxonomies  have  been  constructed  in  the  past  have  been 
undermined.  Integrative  trends  in  developmental,  evolutionary,  and 
cognitive  psychology,  along  with  advances  in  neurology  have 
changed  the  current  picture  of  how  human  thinking  works: 

"Human  reason  depends  on  several  brain  systems,  working 
in  concert  across  many  levels  of  neuronal  organization, 
rather  than  on  a  single  brain  center.  Both  'high-level'  and 
'low-level'  brain  centers,  from  the  prefrontal  cortices  to  the 
hypothalmus  and  brain  stem,  cooperate  in  the  making  of 
reason."  (Damasio  1994) 

Recent  results  in  language  understanding  (e.g.,  Damasio  et  al  1997) 
show  an  extraordinary  and  previously  unsuspected  degree  of 
distribution  of  brain  activity  in  understanding  even  the  simplest 
concepts.  Thus,  when  seen  from  a  neurological  perspective,  what 
we've  previously  thought  about  which  activities  are  similar  to  others 
is  called  seriously  into  doubt. 

The  positive  side  of  this  is  that  as  a  deeper  understanding  of 
cognition  emerges,  the  foundation  upon  which  understanding  of 
human  performance  can  be  built  becomes  more  substantive. 
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SURVEY  STRUCTURE 

i 

We  wished  to  adequately  characterize  team  task  performance  is  such 
a  way  that  training  recommendations  can  be  made.  The  following 
questions  attempt  to  characterize  tasks.  Before  listing  the  questions, 
a  view  of  the  overall  structure  of  the  questions  may  make  the  logic 
easier  to  follow. 

Task  conditions 

Physical  environment 
Information  environment 
Working  environment 
Resources 
Task  structure 

Event  variability,  frequency,  and  gravity 
Attention,  responsibility,  motivation,  vigilance 
Decision  making 
Steps 

Assessment 

Diagnosis 

Prescription/planning 

Execution 

Evaluation 

Approaches,  outcomes,  and  issues 
Methods 
Solutions 
Uncertainty 
Self-management 
Knowledge 
Facts 

Procedures 
Teamwork 
Mutual  monitoring 
Feedback 
Communication 
Reciprocal  backup 
Membership  awareness 
Interdependence 
Shared  skills 

Stability  and  development 
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Errors 

Physical 

Fatigue 

Mental 

Self-management 

Leadership 
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SURVEY  QUESTIONS 


The  survey  questions,  with  some  explanatory  text. 

SECTION  I:  CONDITIONS 

All  jobs  are  done  in  some  kind  of  environment.  For  some  kinds  of 
jobs,  the  environment  is  highly  controlled.  A  surgical  theater  is  an 
example.  In  other  kinds  of  jobs,  there  is  almost  no  control  at  all  of  the 
environment.  Police  work  is  often  like  this.  Other  things  count  as  job 
conditions.  Whether  you  have  to  do  many  jobs  at  once,  how  much 
and  how  reliable  your  information  input  is,  the  physical  environment 
in  which  you  work,  and  whether  you  can  get  the  resources  you 
require  when  you  need  them  are  all  examples  of  work  conditions 
about  which  we  will  ask  questions  in  this  section. 


PHYSICAL  ENVIRONMENT 

In  this  section,  we  will  be  asking  you  questions  about  the  physical 
environment  in  which  you  work.  The  physical  environment  includes 
levels  of  noise,  temperature,  humidity,  and  light  as  well  as  conditions 
that  may  affect  the  environment  such  as  fire,  wind,  earthquakes,  etc. 
Later  in  the  questionnaire  we  will  be  asking  about  other  aspects  of 
your  work  environment  (e.g.,  social,  cultural,  the  work  itself,  the 
sources  and  flow  of  information).  For  now,  please  consider  only  the 
physical  environment  when  you  answer  the  questions  in  this  section. 

1.  To  what  extent  is  the  environment  in  which  the  task  must  be 
performed  physically  stressful  (e.g.,  loud  noise,  challenging  or 
threatening  physical  circumstances)? 

2.  To  what  extent  is  the  environment  in  which  the  task  must  be 
performed  emotionally  stressful  (e.g.,  possible  ambush,  sudden 
catastrophe,  extremely  serious  consequences  of  errors)? 

3.  To  what  extent  is  the  physical  environment  in  which  the  task  to 
be  performed  fast-changing? 
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INFORMATION  ENVIRONMENT 

The  questions  in  this  section  ask  you  about  the  information  you  use 
to  do  your  work.  For  example,  you  may  receive  information  on  paper 
in  the  form  of  reports,  test  results,  memos,  books,  newspapers,  or 
journals.  Or,  you  might  receive  spreadsheets,  text,  data,  or  graphics 
electronically  through  e-mail,  software,  computerized  databases,  or 
the  internet.  You  may  be  given  information  by  other  people  over  the 
phone,  through  radio  transmissions,  or  in  person.  In  some  jobs, 
information  is  largely  received  aurally  (music,  dance,  radio  operator). 
Other  jobs  (e.g.,  dance,  sports,  fire  fighting,  visual  arts,  and  theatre) 
also  depend  on  information  that  is  obtained  visually. 

When  answering  the  questions  in  this  section,  think  of  the  type  of 
information  you  receive  as  a  whole  however  you  put  it  together  from 
whatever  sources  you  have.  In  addition,  we  are  interested  in  your 
information  environment  when  you  are  performing  your  work.  We  are 
not  concerned  with  the  information  you  use  during  training,  learning, 
or  rehearsal. 

4.  To  what  degree  is  information  receipt  (inflow  of  data,  reports, 
updates,  etc.)  reliable?  (This  is  a  question  about  how  easy  it  is  to 
obtain  information,  not  about  its  quality.  For  example,  if  you  have 
a  highly  reliable  supply  of  really  lousy  information,  you  would 
circle  "6"  or  "7".) 

5.  To  what  degree  are  you  swamped  with  information?  (We 
distinguish  between  the  total  flow  of  information  and  the  subset 
of  useful  information.  Here  we  are  concerned  only  with  total 
information  flow.) 

6  To  what  degree  do  you  have  useful,  applicable  information? 

7  To  what  degree  do  you  need  to  examine  your  information  to 
determine  which  of  it  is  valid,  timely,  and  useful? 

8  To  the  extent  that  you  need  to  examine  your  information,  what 
degree  of  skill  is  required  to  do  this  well? 

9  To  what  degree  is  information  normally  adequate  to  your  needs? 
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(Do  you  have  the  right  information?) 

10  To  what  degree  is  information  extraction  from  the  environment 
difficult? 

11  How  complex  is  the  notation  in  which  information  is  presented? 
(Mathematical  and  musical  notations  are  examples  of  complex 
notations.) 


CORE  AND  CONCURRENT  TASK  STRUCTURE 

Some  jobs  require  that  a  person  do  more  than  one  task  at  a  time 
(e.g.,  police  officers  must  concurrently  drive  a  car,  be  aware  of  the 
surroundings,  and  talk  on  the  radio).  Other  jobs,  like  surgery  for 
example,  require  complete  concentration,  and  rule  out  attending  to 
other  tasks  at  the  same  time.  We'll  now  ask  you  questions  about  the 
number  of  tasks  that  must  be  done  concurrently.  These  are  not 
subtasks  that  must  be  attended  to  alternately,  switching  from  one  to 
the  other.  Instead,  we  are  concerned  with  subtasks  that  must  be 
done  at  exactly  the  same  time. 

In  the  following  questions,  when  we  talk  about  "core  tasks"  we  mean 
any  activities  that  contribute  to  the  achievement  of  your  main  goal(s). 
We  distinguish  these  from  non-core  tasks:  Any  tasks  that  are 
peripheral  or  unrelated  to  the  accomplishment  of  the  most  important 
goal(s)  of  the  work. 

12  Based  on  your  own  subjective  perception  of  your  work,  how 
many  core  tasks  [those  associated  with  or  required  for 
accomplishment  of  your  main  goal(s)]  need  to  be  done  at  one 
time? 

13  To  the  extent  that  must  do  many  things  at  once  to  achieve  your 
goal(s),  to  what  degree  do  the  demands  of  the  various  subtasks 
conflict? 

14  To  what  degree  are  there  other  tasks,  distinct  from  the  core 
task(s),  that  must  be  performed  at  the  same  time  as  the  the  core 
task?  A  "distinct  task"  is  any  that  is  peripheral  or  unrelated  to 
the  accomplishment  of  the  most  important  goal(s)  of  the  work. 
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For  example,  a  firefighter  might  have  to  talk  with  reporters  while 
fighting  a  fire.  By  contrast,  a  surgeon  would  not  have  to  talk  to 
reporters  while  performing  surgery. 

15  To  the  extent  that  there  are  other  non-core  tasks,  to  what 
degree  do  the  demands  or  goals  of  these  other  tasks  conflict 
with  the  core  task? 


RESOURCES 

Effective  job  performance  requires  particular  resources  be  made 
available  to  the  worker.  For  example,  fire  fighters  need  the 
appropriate  vehicles,  manpower,  equipment,  water,  and  chemicals  to 
put  out  fires.  Musicians  require  performance  space  and  instruments 
to  perform.  Similarly,  dancers  need  appropriate  costumes,  shoes,  and 
physical  space  for  their  performances.  In  this  section,  we  are  asking 
about  the  resources  required  for  your  job  performance.  When 
answering  the  following  questions,  do  not  consider  the  resources 
required  to  prepare  to  perform  your  job,  such  as  rehearsal  space, 
training  tools,  educational  materials,  practice  time,  or  financial 
support  that  contributes  to  learning  your  trade.  We  ask  about  only 
the  resources  required  to  do  the  actual  work. 

16  To  what  degree  is  access  to  resources  varied  and  somewhat 
uncertain? 

17  To  what  degree  do  task  elements  require  unique  resources,  in 
contrast  with  having  many  alternatives? 
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SECTION  II:  STRUCTURE 


The  questions  in  Section  II,  Structure,  concern  the  external  factors 
that  influence  how  your  work  is  organized.  In  this  section  we  make  a 
distinction  between  "event  oriented"  and  "continuous"  task  domains. 
The  structure  of  some  kind  of  work,  firefighting  for  example,  is 
determined  by  the  nature  of  external  events  (in  this  example,  fires  or 
toxic  spills).  Other  kinds  of  work  are  better  characterized  as 
ongoing,  continuous  processes.  Building  construction  is  an  example 
of  a  continuous  process.  Note  that  a  process  may  go  through  distinct 
phases  and  still  be  considered  continuous.  Also  note  that  an  event- 
oriented  domain  may  feel  "continuous",  or  at  least  highly  predictable 
(regular  Sunday  performances  for  a  church  choir,  for  instance.) 

Some  kinds  of  work  combine  both  events  and  continuous  processes. 
Nursing  is  an  example. 


EVENT  EXTENT  AND  VARIABILITY 

If  the  events  you  encounter  in  your  work  are  very  different  from  each 
other,  then  they  can  be  said  to  be  variable.  For  example,  an  event 
that  some  police  officers  face  is  a  domestic  dispute,  which  is  very 
different  from  stopping  a  drunk  driver  or  arresting  a  graffiti  artist.  In 
this  case,  police  work  is  best  described  as  highly  variable.  The 
questions  in  this  section  of  the  survey  ask  about  the  variability  of 
events  in  your  work. 

18  To  what  degree  are  task  events  (episodes)  variable? 

19  To  the  extent  that  there  is  variability  among  events,  to  what 
degree  is  the  difference  one  of  size  and/or  complexity? 

20  If  there  is  variability  between  events,  to  what  degree  is  the 
difference  one  of  potential  seriousness? 


EVENT  FREQUENCY 

In  this  section,  we  are  interested  in  the  frequency  of  events.  For 
some  jobs,  events  are  relatively  infrequent,  e.g.,  major  canyon  fires 
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in  fire  fighting.  For  other  jobs,  events  occur  very  frequently,  e.g., 
traffic  offenses  in  law  enforcement.  In  the  following,  we  ask  you 
about  the  frequency  of  events  in  your  work. 

21  To  what  degree  are  there  common  categories  of  events? 

22  If  there  are  wide  variations  among  events,  to  what  degree  are 
"minor"  task  episodes  frequent?  (If  there  are  not  wide 
variations,  please  mark  this  question  "N/A".) 

23  If  there  are  wide  variations  among  events,  to  what  degree  are 
"major"  task  episodes  frequent?  (If  there  are  not  wide 
variations,  please  mark  this  question  "N/A".) 

24  If  there  are  not  wide  variations  among  events,  to  what  degree 
are  are  task  episodes  in  general  frequent 

EVENT  GRAVITY  &  CONSEQUENCES 

The  seriousness  of  errors  associated  with  events  varies.  The 
consequences  of  making  a  mistake  while  flying  a  fighter  airplane 
during  a  dogfight  can  be  severe,  while  the  consequences  of 
incorrectly  filling  out  a  traffic  citation  form  are  less  so.  This  section 
contains  questions  regarding  the  consequences  of  errors  in  events. 

25  How  serious  are  the  consequences  of  errors? 

26  To  what  degree  can  errors  normally  be  amended  or  compensated 
for  by  future  actions? 


ALLOCATING  ATTENTION  AND  RESOURCES 

When  your  work  is  a  more  or  less  continuous  process,  knowing  what 
task  to  attend  to  and  spend  time  working  on  is  a  big  part  of  many 
jobs.  In  some  work  domains,  team  members  must  know  how  to 
allocate  both  their  own  time  as  well  as  available  resources.  This 
section  of  the  survey  contains  questions  that  address  the  allocation 
of  worker's  attention  and  resources. 
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27  To  what  degree  does  good  job  performance  require  knowing 
how  to  effectively  allocate  your  attention  to  a  variety  of  tasks? 

28  To  what  degree  does  good  job  performance  require  the  ability  to 
set  effective  priorities? 

29  To  what  degree  must  you  anticipate  what  will  be  needed  and 
make  small  predictions  in  your  work? 

30  How  important  is  knowing  how  to  allocate  your  physical  or 
material  resources? 

31  How  often  do  you  typically  need  to  reconsider  or  reallocate  your 
physical  or  material  resources? 

32  What  degree  of  reallocation  or  redirection  of  physical  or  material 
resources  is  common? 

33  How  important  is  knowing  how  to  allocate  your  personnel 
resources? 

34  How  often  do  you  typically  need  to  reconsider  or  reallocate  your 
personnel  resources? 

35  What  degree  of  reallocation  or  redirection  of  personnel  resources 
is  common? 


RESPONSIBILITY 

Knowing  what  you're  responsible  for,  what  the  team  is  responsible 
for,  when  subtasks  need  to  be  completed,  and  completing  them  on 
time  are  important  components  of  many  jobs.  In  some  cases, 
determining  a  deadline  for  completing  a  task  depends  on 
understanding  how  that  task  fits  into  the  big  scheme  of  the  work  and 
how  it  contributes  to  the  overall  goal.  The  questions  in  this  section 
pertain  to  workers'  responsibilities,  with  an  emphasis  on  timing  and 
coordination. 

36  How  important  is  it  for  each  team  member  to  understand  how 
their  work  contributes  to  the  overall  goal,  i.e.,  how  it  fits  into  the 
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"big  picture"? 

37  To  what  degree  can  you  and  other  team  members  alter  and 
adjust  your  work  to  better  fit  the  current  situation  and  needs? 

38  To  what  degree  is  estimating  how  long  it  will  take  to  complete  a 
task  part  of  the  job? 

39  How  severe  are  the  consequences  of  not  completing  a  task  on 
time? 

40  How  important  is  knowing  when  to  perform  tasks? 

41  To  what  degree  is  your  work  interwoven  with  that  of  your 
teammates?  Said  another  way,  how  sensitive  is  your  work  to 
everyone  on  the  team  being  in  sync,  completing  subtasks  on  time 
and  in  the  right  order,  and  generally  being  connected? 


MOTIVATION 

Motivation  is  usually  a  critical  factor  in  job  performance.  Maintaining 
high  levels  of  motivation  can  be  particularly  important  when  the  work 
must  be  conducted  over  a  long  period  of  time.  For  example,  the 
extended  durations  of  product  development  projects,  typically  many 
months  to  many  years,  may  make  it  difficult  for  team  members  to 
maintain  high  levels  of  motivation. 

42  How  motivated,  typically,  are  team  members  in  your  work  over 
the  entire  duration  of  the  work? 

43  To  what  degree  does  motivation  vary  over  the  full  course  of  job 
performance? 

44  To  what  degree  does  the  continuous  nature  of  the  work  make  it 
difficult  to  maintain  a  high  level  of  motivation? 

45  To  what  degree  do  team  members  in  your  kind  of  work  typically 
mutually  support  or  reinforce  maintaining  high  levels  of 
motivation  for  the  team  as  a  whole? 
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46  To  what  degree  does  motivation  depend  on  job  rewards  (salary, 
benefits,  perks,  etc.)? 

47  To  what  degree  do  job  rewards  (salary,  benefits,  perks,  etc.) 
depend  on  high  levels  of  motivation? 


EXTENDED  VIGILANCE 

Vigilance  is  the  ability  to  maintain  one's  attention  on  a  specific  subject 
or  task  over  extended  periods  of  time.  For  example,  radar  operators 
spend  much  of  their  work  day  looking  at  a  radar  display  screen, 
attempting  to  detect  anomalous  events.  It  is  not  surprising  that 
heavy  vigilance  loads  often  produce  fatigue.  The  questions  in  this 
section  are  about  the  vigilance  demands  of  your  job. 

48  To  what  degree  does  your  work  require  extended  vigilance,  as 
defined  above? 

49  To  what  degree  is  vigilance  distributed  and  traded  off  amongst 
team  members?  In  other  words,  does  everyone  have  to  be 
equally  vigilant  at  all  times,  or  does  the  locus  of  vigilance  shift 
among  team  members  during  task  performance? 

50  To  what  degree  does  required  vigilance  produce  fatigue  in  your 
job? 
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SECTION  III:  DECISION  MAKING 


The  third  section  of  the  survey  is  about  decision  making.  Jobs 
typically  involve  one  or  more  components  of  decision  making: 
assessment,  diagnosis,  prescription  or  planning,  execution,  and 
evaluation.  Some  jobs  require  skill  in  each  of  these  elements. 

We  are  interested  in  the  performance  of  your  work,  in  contrast  to  any 
preparation  for  performance  you  may  do.  For  example,  performing 
arts  typically  involve  assessment,  diagnosis,  and  planning  in  the 
rehearsal  phase,  but  not  in  the  actual  performance.  By  then,  the 
decisions  have  typically  been  made  and  all  that  is  left  is  execution 
and  evaluation  of  the  performance.  Other  jobs,  e.g.,  a  physician's 
practice,  involve  all  five  elements  throughout  job  performance. 

We  want  to  ask  you  questions  about  the  elements  that  are  part  of 
your  job.  To  do  that,  we  need  to  specify  which  of  the  five  elements 
listed  above  (assessment,  diagnosis,  prescription  or  planning, 
execution  and  evaluation)  is  part  of  your  work.  Please  indicate  which 
of  the  elements  are  involved  in  your  work  by  checking  the 
appropriate  boxes. 

Assessment 

Assessment  is  finding  out  what  is  going  on:  Identifying  the  status  or 
state  of  the  important  aspects  of  a  situation.  For  example,  physicians 
assess  a  new  case  (situation)  by  finding  out  what  symptoms  an  ill 
patient  is  experiencing.  Similarly,  fire  fighters  gather  information 
about  wind,  extent  of  fire,  substance  being  burned,  etc.  in  assessing 
a  fire.  Assessment  can  be  thought  of  as  information  or  data 
gathering  in  the  initial  phases  of  dealing  with  a  situation.  It  is 
different  that  diagnosing  the  problem.  Diagnosis  is  concerned  with 
determining  the  source  of  the  problem  (and  will  be  covered  next). 
Please  note  that  we  are  not  interested  in  assessment  that  occurs 
during  practice,  rehearsal,  or  job  training.  We  are  only  interested  in 
assessment  that  occurs  during  actual  job  performance. 

Does  your  job  performance  involve  assessment?  If  so,  please  check 
the  following  box. 

_ Yes,  a  significant  part  of  my  job  involves  assessment. 
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Diagnosis 

Diagnosis,  as  indicated  above,  amounts  to  identifying  the  cause  of 
the  problem.  After  gathering  sufficient  data  in  the  assessment 
phase,  doctors  diagnose  diseases.  The  fire  fighter  will  also  gather 
necessary  information  to  determine  the  source  of  the  fire,  which  is 
important  to  knowing  how  to  attack  it.  Does  the  fire  involve 
hazardous  chemicals,  is  it  likely  to  move  fast,  what  direction  wjll  it 
take?  Again,  we  are  only  interested  in  job  performance,  not  training, 
education,  or  rehearsal. 

Does  your  job  involve  diagnosis?  If  so,  please  check  the  following 
box. 

_ Yes,  a  significant  part  of  my  job  involves  diagnosis. 


Prescription  and  Planning 

The  third  element  of  decision  making  is  to  create  a  prescription  or 
plan  for  treating  a  problem.  In  this  phase,  the  physician  decides 
what  drug  or  other  therapy  to  give  a  patient.  The  fire  fighter 
determines  how  he  or  she  will  contain  and  stop  the  fire  and  what 
resources  to  use  in  doing  so.  Again,  we  are  only  interested  in  job 
performance,  not  training,  education,  or  rehearsal. 

Does  job  performance  in  your  field  involve  planning  and  prescription? 
If  so,  please  check  the  following  box. 

_ Yes,  a  significant  part  of  my  job  involves  planning  and/or 

prescription. 

Execution 

All  work  endeavors  involve  some  type  of  performance.  Execution,  the 
fourth  phase  of  decision  making,  is  when  that  performance  takes 
place.  A  doctor  might  perform  a  surgery  or  administer  the  prescribed 
therapy.  Fire  fighters  would  distribute  resources  (water,  chemicals, 
vehicles,  manpower,  etc.)  to  fight  the  fire. 

There  are  wide  variations  in  execution.  For  example,  if  your  job 
involves  generating  a  plan  but  other  people  actually  execute  it,  then 
the  execution  portion  of  your  job  may  be  extremely  simple,  and  you 
would  not  answer  the  questions  in  this  section.  On  the  other  hand,  if 
significant  skill  is  required  for  good  execution  in  your  work  (if  some 
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people  are  clearly  better  at  It  than  others),  then  you  should  answer 
the  questions  in  this  section. 

Does  your  job  involve  significant  execution?  If  so,  please  check  the 
following  box. 

_ Yes,  a  significant  part  of  my  job  involves  execution. 

Evaluation 

Evaluation  is  sometimes  performed  after  a  job  is  finished.  In  this 
phase  of  decision  making,  workers  evaluate  how  well  and/or  how 
poorly  they  did  on  various  aspects  of  the  job.  Sometimes  the 
evaluation  is  done  by  supervisors  and  sometimes  it  is  completed  by 
the  workers  themselves. 

We  are  interested  here  in  post-action  evaluation.  In  most  work, 
there  is  another  kind  of  evaluation  that  goes  on  all  the  time,  namely 
checking  whether  appropriate  progress  is  being  made  toward  the 
goal.  We  will  ask  you  about  that  later,  but  for  now  we  are  just 
interested  in  post-action  evaluation. 

Must  you  or  do  you  typically  evaluate  job  performance  when  it  is 
completed?  If  so,  please  check  the  following  box. 

_ Yes,  a  significant  part  of  my  job  involves  evaluation. 


TASK  PROCESSES:  ASSESSMENT 

In  this  section,  we  are  concerned  only  with  assessing  a  task 
situation— determining  what  is  going  on.  We  defer  consideration  of 
diagnosis  (understanding  the  causal  source  of  the  situation)  to  the 
next  section.  We  know  it  is  often  hard  to  draw  a  sharp  line  between 
assessment  and  diagnosis,  but  ask  that  you  attempt  to  do  so  as  you 
answer  the  following  questions. 

51  What  level  of  skill  does  assessment  require? 

52  To  what  degree  are  the  surface  features  indicative  of  the  deep 
structure?  Said  another  way,  is  what  you  see  really  what  there 
is,  or  do  you  have  to  look  below  the  surface? 
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53  To  what  degree  are  there  significant  irrelevant  or  deceptive 
features,  things  that  are  easy  to  see  but  are  misleading  about 
the  real  situation? 

54  To  what  degree  do  small  details  influence  assessment?  Said 
another  way,  is  the  big  picture  the  main  thing,  or  can  the  entire 
situation  change  with  a  single  detail? 

55  To  what  degree  are  misleading  or  confusing  pieces  of  evidence 
the  result  of  antagonistic  intention?  Is  it  necessary  to  consider 
evidence  with  suspicion  of  intentional  deception? 

56  To  what  degree  is  there  typically  more  than  one  way  to  frame  a 
situation?  "Framing"  is  just  the  way  you  look  at  a  problem.  For 
some  tasks,  the  way  to  look  at  a  situation  is  clear  right  away.  For 
other  tasks,  figuring  out  which  way  to  look  at  a  situation  is  a  big 
part  of  assessing  it. 

57  With  what  degree  of  frequency  are  avoidable  mistakes  made  in 
assessing  a  situation? 

58  To  what  extent  can  probes  be  made  to  confirm/disconfirm  an 
assessment?  A  probe  is  simply  any  method  for  testing  whether 
an  assessment  is  correct  prior  to  starting  work. 

59  How  often  do  you  have  to  choose  a  course  of  action  despite  not 
having  a  complete  assessment? 


TASK  PROCESSES:  DIAGNOSIS 

As  noted  previously,  diagnosis  is  identifying  the  cause  of  the  problem. 
Physicians  identify  the  physical  problem  or  disease,  and  police  officers 
determine  if  an  erratic  driver  is  driving  under  the  influence.  If  your 
job  performance  does  not  involve  a  diagnosis  please  do  not  answer 
the  following  questions.  We  are  concerned  only  with  job 
performance,  not  training,  education,  or  rehearsal. 

60  What  level  of  skill  does  diagnosis  require? 
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61  To  what  degree  is  the  evidence  upon  which  a  diagnosis  is  based 
frequently  or  importantly  ambiguous? 

62  To  what  degree  must  multiple  hypotheses  be  considered 
concurrently?  Said  another  way,  are  there  many  different 
reasons  that  a  situation  could  be  as  it  Is? 

63  To  what  degree  is  there  typically  more  than  one  way  to  frame  a 
diagnosis  (e.g.,  to  decide  which  factors  are  significant  in  a 
particular  case)? 

64  With  what  degree  of  frequency  are  avoidable  diagnostic  mistakes 
made? 

65  To  what  degree  is  there  well-established  theory  or  guidelines  for 
most  diagnostic  decisions? 

66  To  the  extent  that  there  is  an  established  theory  or  set 
guidelines  for  diagnosis,  to  what  degree  is  application  of  the 
theory  or  guidelines  straightforward? 

67  To  what  degree  is  quickness  of  diagnosis  required?  How  quickly 
must  a  diagnosis  be  made? 

68  To  what  degree  is  deferring  diagnosis  in  itself  a  significant 
action? 

69  To  what  degree  can  you  be  certain  of  your  diagnoses? 

70  To  what  extent  can  probes  be  made  to  confirm/disconfirm 
diagnoses? 


TASK  PROCESSES:  PLANNING  &  PRESCRIPTION 

The  following  set  of  questions  pertain  to  the  development  of  a 
prescription  or  plan  for  treating  a  problem.  Again,  we  are  only 
interested  in  job  performance/not  training,  education,  or  rehearsal. 
If  your  job  does  not  require  planning  or  prescription  during  job 
performance,  do  not  answer  the  following  questions. 
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71  What  level  of  skill  does  prescription  (deciding  what  to  do) 
require? 

72  What  level  of  skill  does  planning  (deciding  how  to  do  it)  require? 

73  Once  a  diagnosis  has  been  made,  how  many  actions  can  be 
prescribed? 

74  To  what  degree  is  planning  or  prescribing  separate  from  and 
prior  to  execution,  or  is  it  normal  to  plan  a  little,  execute  a  little, 
plan  a  little,  etc.? 

75  To  what  degree  are  prescriptions  and/or  plans  explicit  or  tacit? 

76  To  what  degree  are  changes  in  prescription  or  plan  explicit  or 
tacit? 

77  To  what  degree  does  the  evolution  of  plans  vary  between 
situations? 


TASK  PROCESSES:  EXECUTION 

Execution  is  actually  doing  the  work  involved  in  your  job.  The 

following  set  of  questions  ask  about  various  issues  associated  with 

the  execution  of  job  performance. 

78  What  level  of  skill  does  execution  (actually  implementing  the 
prescription  or  plan)  require? 

79  To  what  degree  must  multiple  paths  be  maintained  as 
possibilities  during  execution? 

80  To  the  extent  that  multiple  paths  must  be  maintained,  to  what 
degree  is  the  task  leader  mainly  responsible  for  this,  or  must  all 
team  members  be  cognizant? 

81  How  important  is  timing  in  task  execution?  For  example,  for  a 
musician,  very  small  differences  in  when  an  entry  is  made  have 
major  effects  on  the  quality  of  the  performance.  By  contrast,  for 
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an  accountant,  finishing  the  accounts  by  the  end  of  the  month 
may  be  the  goal  and  the  details  of  timing  insignificant. 


TASK  PROCESSES:  EVALUATION 


The  following  section  contains  question  regarding  the  evaluation  of 
work  completed.  We  ask  about  a  variety  of  issues  related  to  the 
evaluation  of  work. 

82  How  important  a  skill  is  post-event  evaluation? 

83  To  what  degree  is  evaluation  formally  encouraged  or  required? 

84  To  what  degree  is  evaluation  restricted  by  politics,  reputations, 
or  other  constraints? 


TASK  PROCESSES:  TASK  METHODS 

Based  on  your  answers  in  the  previous  section,  we  assume  that  your 
work  involves  problem  solving  during  task  execution.  Specifically,  if 
you  marked  that  you  must  assess,  diagnose,  and  prescribe  (or  plan) 
before  doing  the  actual  work  during  task  performance,  then  "problem 
solving"  is  part  of  your  work.  The  following  set  of  questions  concern 
task-specific  problem  solving  methods  you  may  apply  in  your  work. 

85  To  what  degree  are  methods  for  problem-solving 
standardized? 

86  To  the  extent  that  there  are  standard  methods,  how  often  are 
they  actually  applied  in  the  field? 

87  To  what  degree  can  decisions  or  actions  normally  be 
compensated  for  by  future  actions? 

88  How  significant  to  problem-solving  is  predicting  the  behavior  of 
others? 

89  How  many  tries  does  one  normally  have  in  attempting  to  solve  a 
problem? 


Syukhtun  Research,  Inc. 


53 


90  To  what  extent  can  you  experiment  while  implementing  a 

solution  or  plan?  For  example,  in  packing  items  into  boxes,  many 
arrangements  might  be  tried.  In  building  skyscrapers,  on  the 
other  hand,  the  plans  must  be  followed. 


TASK  PROCESSES:  SOLUTION  CHARACTERISTICS 

As  in  the  previous  section  (task  methods),  we  assume  that  your  job 
performance  involves  problem  solving  as  part  of  task  execution.  This 
section  of  the  survey  contains  question  about  the  solutions  that  are 
developed  as  part  of  the  problem  solving  effort.  Solutions  are  the 
ways  in  which  you  treat  the  situation  or  problem.  A  mathematical 
solution  would  be  an  answer  to  a  set  of  equations.  A  fire  fighting 
solution  would  be  the  certified  end  to  a  fire.  And  so  on.  Clearly, 
some  kinds  of  tasks  have  more  clearly  defined  solutions  than  others. 

91  How  long  after  a  solution  is  implemented  is  it  apparent  that  it 
was  correct  or  incorrect? 

92  How  frequently  can  solutions  be  proven  correct? 

93  To  what  degree  can  you  trust  solutions  on  their  appearance?  For 
example,  the  multimillion  dollar  Oakland  fire  grew  from  a  small  fire 
that  appeared  to  have  been  successfully  put  out. 

94  How  often  are  solutions  only  be  known  to  be  satisfactory  after 
they've  been  completely  implemented? 

95  How  much  do  solutions  vary  in  quality? 

96  To  what  degree  is  it  necessary  to  trade  off  the  quality  of  a 
solution  with  other  factors  when  constructing  a  solution? 

97  To  what  degree  do  solutions  have  to  satisfy  multiple,  not 
necessarily  complimentary  criteria? 

98  How  frequently  does  task  performance  require  proceeding 
despite  the  presence  of  only  a  partial  solution? 
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TASK  PROCESSES:  UNCERTAINTY 


Uncertainty  is  a  factor  in  many  kinds  of  work.  For  example,  police 
officers  never  know  what  they  will  encounter  on  the  street  during 
their  shift.  At  the  other  extreme,  performers  are  typically  well 
rehearsed  such  that  very  little  is  uncertain  (other  than  crowd 
reaction!).  Most  steps  are  executed  as  planned.  The  following  set  of 
questions  asks  about  the  degree  of  uncertainty  and  how  it  is 
managed  in  your  job. 

99  To  what  degree  are  parts  of  the  task  intrinsically  or  unavoidably 
uncertain? 

100  How  often  is  significant  data  uncertain? 

101  How  often  is  knowledge  incomplete  or  uncertain? 

102  What  degree  of  uncertainty  is  typical  in  deciding  how  to  think 
about  a  particular  situation? 


TASK  PROCESSES:  SELF-MANAGEMENT 

Self  management  is  the  continuous  monitoring  of  one  owns 
performance.  Some  jobs  require  accurate  and  continuous  self 
monitoring,  others  do  not.  For  example,  dancers  must  continuously 
monitor  their  placement  in  time  and  location  relative  to  other  dancers 
and  to  the  music.  Soldiers,  fire  fighters,  and  police  must  continuously 
evaluate  their  effectiveness  during  their  work.  The  following 
questions  ask  about  the  degree  to  which  self-monitoring  (also  called 
self-management  or  metacognition)  is  important  to  your  work. 

103  How  important  is  self-monitoring  of  decision  and  performance 
timing?  For  example,  for  some  tasks,  it  is  more  important  to  make 
a  decision  at  the  right  time  than  it  is  to  wait  until  all  of  the 
possible  data  has  come  in. 

104  How  important  is  self-monitoring  of  fatigue  or  distraction  as 
these  affect  task  performance? 


Syukhtun  Research,  Inc. 


56 


105  How  important  is  self-monitoring  of  process  involvement  as  this 
affects  task  performance?  (Involvement  issues  might  include 
maintaining  a  positive  attitude,  maintaining  consistency, 
professionalism,  or  concentration.) 

106  To  what  degree  does  task  performance  require  frequents  shifts 
of  attention? 

107  To  what  degree  is  the  team  or  team  leader  frequently 
interrupted  by  external  elements  that  are  not  task-related  during 
task  performance? 

108  How  difficult  is  it  to  maintain  the  thread  of  concentration  while 
dealing  with  interruptions?  In  some  tasks,  regaining 
concentration  after  an  interruption  is  not  hard.  In  others, 
especially  when  many  balls  must  be  kept  in  the  air  at  once,  it  can 
be  very  difficult. 


KNOWLEDGE:  FACTS 

This  section  of  the  survey  concerns  the  knowledge  a  worker  must 
possess  to  perform  his/her  work  effectively.  Two  types  of  knowledge 
are  distinguished:  knowledge  of  facts  and  information,  and 
knowledge  of  procedures.  Knowledge  of  facts  is  the  ability  to  recall, 
recognize,  or  otherwise  use  information  pertinent  to  the  job. 
Knowledge  of  procedures  is  knowing  how  to  do  the  task.  The 
following  questions  are  about  knowledge  about  facts. 

109  How  large  is  the  volume  of  facts  to  be  known  or  recalled? 

110  To  what  degree  are  there  many  facts  that  might  apply  to  a 
situation,  and  part  of  knowing  the  job  is  to  recall  just  the 
relevant  ones? 

111  How  accurately  must  facts  be  recalled? 

112  How  quickly  must  facts  be  recalled? 

113  To  what  degree  do  facts  change  over  time?  There  are  two 
reasons  facts  can  change.  One  is,  the  state  of  affairs  they 
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describe  changes.  For  example,  a  treaty  may  expire,  a  market 
may  close,  or  a  resource  may  become  accessible.  The  second 
reason  is  that  the  relevance  of  a  "bundle"  of  facts  may  change. 
For  instance,  an  artillery  commander  needs  to  know  the  range 
and  accuracy  of  the  enemy's  guns.  If  the  opponent  changes  to  a 
new  gun  and  eliminates  the  old,  then  the  facts  about  enemy  gun 
range  and  accuracy  may  change  completely.  (Even  though  the 
old  facts  are  still  true,  they  are  no  longer  relevant.) 

114  What  proportion  of  facts  must  simply  be  known,  in  contrast  with 
facts  that  can  be  inferred  with  reasonable  accuracy?  For 
example,  you  either  know  your  mother's  maiden  name  or  you 
don't.  It  can't  be  guessed  based  on  other  evidence.  On  the  other 
hand,  when  a  fly  ball  will  hit  the  ground  can  be  guessed  pretty 
accurately. 


KNOWLEDGE  OF  PROCEDURES 

A  procedure  is  a  step  by  step  sequence  of  actions  that  must  be 
performed  to  complete  a  task.  As  noted  in  the  previous  section, 
procedural  knowledge  involves  knowing  how  to  execute  task  steps. 
For  example,  maintaining  a  valve  in  a  power  plant  typically  requires 
dismantling,  inspecting,  cleaning,  repairing  components,  replacement 
of  parts,  and  putting  the  valve  back  together.  Together,  the  task 
steps  constitute  a  procedure,  for  which  skill  is  required  at  just  about 
every  step  along  the  way.  Knowing  how  to  execute  each  component 
is  procedural  knowledge.  The  following  questions  ask  about 
procedural  knowledge  that  may  be  relevant  to  your  work. 

115  To  what  degree  are  step  by  step  procedures  part  of  the  task? 

116  To  what  degree  does  good  performance  require  selecting  the 
right  procedure  from  among  many  possible  procedures  for  the 
particular  circumstances?  That  is,  figuring  out  the  right  thing  to 
do? 

117  What  level  of  skill  does  the  execution  of  procedures  require?  That 
is,  once  you  know  what  to  do,  does  it  take  a  lot  of  skill  to  actually 
do  it? 


Syukhtun  Research,  Inc. 


58 


118  To  what  degree  does  one  have  to  anticipate  the  next  step  when 
performing  a  procedure? 

119  To  the  extent  that  anticipation  is  important,  to  what  degree  does 
that  include  knowing  how  others  are  going  to  behave  or  react 
and  adjusting  accordingly? 

120  To  what  degree  is  having  the  right  attitude  an  important  part  of 
performing  task  procedures?  For  example,  in  some  tasks,  stress, 
or  physical  or  emotional  fatigue  tends  to  lead  to  reduced  efforts. 

121  To  what  degree  do  the  procedures  have  a  substantial  physical 
component?  Examples  include  marksmanship,  surgery,  and 
musical  performance.  Accounting,  by  contrast,  does  not  make 
strong  demands  on  physical  skill. 
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SECTION  IV:  TEAMWORK 


This  section  of  the  survey  asks  questions  about  how  teams  operate 
in  your  work.  The  eight  subsections  address  different  aspects  of 
teamwork  including  communication,  monitoring  teammates 
performance,  evaluative  feedback,  backup,  team  membership, 
interdependence,  shared  skills,  and  the  stability  of  the  team.  When 
answering  the  questions  in  this  section,  please  consider  how  the 
team  functions  as  a  whole.  We  are  not  so  interested  in  any  single 
team  member's  role  in  the  team  or  how  that  individual  interacts  with 
his/her  teammates.  Instead,  we  want  to  know  how  all  of  the 
members  interact  in  general. 


TEAM  MUTUAL  MONITORING 

Some  teamwork  requires  that  team  members  closely  monitor  each 
others  performance  on  the  job.  In  some  cases,  continuous  monitoring 
is  critical  to  the  task  because  the  actions  executed  by  one  team 
member  entirely  depend  on  another  teammate's  work.  For  example, 
to  make  a  shot  in  basketball  players  must  be  passed  the  ball  by  their 
teammates  at  an  appropriate  point  in  the  game.  In  this  case,  both 
the  recipient  of  the  pass  and  the  passer  must  be  continuously 
monitoring  each  other's  and  other  players'  locations.  Sometimes 
members'  roles  are  more  independent,  and  can  be  performed  in 
isolation  without  much  monitoring  of  teammates,  e.g.,  factory  worker 
roles  in  an  assembly  line.  In  this  section,  we  ask  questions  regarding 
the  degree  of  team  monitoring  necessary  to  your  job. 

122  How  important  is  monitoring  teammates  behavior  to  task 
performance? 

123  To  what  degree  are  teammates  expected  to  monitor  each  others' 
actions  for  correctness  and/or  timeliness? 

124 To  what  degree  must  teammates  know  whether  teammates  are 
in  need  of  assistance? 

125  To  what  degree  do  team  members  monitor  whether  appropriate 
progress  is  being  made  toward  the  goal  of  the  team? 
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126  How  difficult  is  it  to  monitor  teammates'  performance? 

127  To  what  degree  does  rapid  change  in  the  situation  affect 
teammates'  ability  to  monitor  one  another? 


TEAM  FEEDBACK 

Feedback  is  information  that  a  worker  receives  about  the  quality  and 
adequacy  of  his  job  performance.  Sometimes  feedback  is  provided  by 
equipment  or  the  environment  is  which  the  task  is  being  performed. 
For  example,  a  musician  immediately  knows  by  the  sound  of  the 
instrument  whether  he/she  has  correctly  performed  a  musical  piece. 

In  other  cases,  feedback  is  provided  by  team  members.  The 
questions  in  this  section  addresses  the  type  of  feedback  that  is  given 
to  team  members  by  their  teammates  or  leaders.  As  such,  the  type  of 
feedback  we  are  concerned  with  is  a  form  of  communication  among 
the  individuals  on  a  team.  When  answering  the  questions  in  this 
section,  please  think  of  how  teammates  communicate  information 
about  each  other's  performance. 

128  How  important  is  giving  feedback  to  teammates  to  effective  task 
activity? 

129  How  important  is  receiving  feedback  from  teammates  to  task 
activity? 

130  To  what  degree  is  feedback  used  during  task  performance  to 
keep  performance  on  track? 

131  To  what  degree  is  feedback  direct  among  team  members? 

132  To  what  degree  is  feedback  mediated,  e.g.,  through  a  mutual 
superior  or  in  some  other  way? 

133  To  what  degree  is  the  flow  of  feedback  typically  influenced  by 
differences  in  status,  tenure,  or  rank  among  team  members? 

134  How  explicit  is  feedback?  Or,  is  it  implicitly  communicated  (e.g.. 
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via  attitude,  degree  of  cooperation,  physical  postures,  etc.)? 

135  To  what  degree  might  team  members  expect  reprisal  for  giving 
unwanted  feedback? 

136  How  formalized  are  the  procedures  for  giving  and  receiving 
feedback? 


COMMUNICATION 

Communication  among  team  members  can  be  verbal  or  non  verbal.  In 
either  case,  information  about  the  work  itself  or  about  other 
teammates  is  conveyed.  This  section  contains  just  three  questions 
about  how  communication  occurs  among  teammates  in  your  work. 

137  To  what  degree  is  communication  among  team  members  an 
"open  channel"? 

138  To  what  degree  is  there  a  strong  communication  requirement  to 
"close  the  loop"  (confirm  correctness  or  appropriateness)  during 
task  performance? 

139  How  much  task-specific  jargon  or  non-verbal  language  is  needed 
to  effectively  communicate? 


TEAM  RECIPROCAL  BACKUP 

Teamwork  often  entails  "backing  up"  a  teammate's  work  when  his  or 
her  performance  drops  or  when  it  is  not  possible  for  a  single 
individual  to  complete  the  work.  For  example,  a  soldier  might  take 
over  another  soldier's  responsibilities  when  an  injury  occurs. 
Teammates  may  simply  provide  assistance  during  backup,  or  they 
may  completely  take  over  the  task.  Reciprocal  backup  refers  to  the 
degree  to  the  team  members  provide  backup  for  each  other.  This 
section  contains  questions  about  reciprocal  backup  in  teams  you 
belong  to  as  part  of  your  work. 

140  To  what  degree  is  reciprocal  backup  an  important  part  of  task 
performance? 
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141  How  important  to  confidant  task  performance  is  knowing  that 
one  will  be  backed  up  ? 

142  How  common  is  actual  active  backup  in  the  field? 

143  How  welcome  are  team  members  to  jump  into  other  team 
members'  subtasks  if  they  perceive  a  need? 

144  How  comfortable  are  team  members  typically  in  accepting  others' 
assistance? 

145  To  what  degree  is  there  a  common  set  of  guidelines,  practices,  or 
conditions  where  backup  is  called  for  and  expected? 

146  How  frequently  do  team  members  have  to  anticipate  others' 
need  for  backup? 

147  To  what  degree  do  team  members  have  to  change  their 
performance  to  help  teammates  avoid  a  future  need  for  backup? 


TEAM  MEMBERSHIP  AWARENESS 

This  section  of  the  survey  contains  questions  about  team  spirit  and 
the  degree  to  which  team  members  have  a  shared  awareness  of  the 
team  as  a  single  unit.  In  many  situations,  it  is  very  important  that  the 
team  members  develop  a  sense  of  working  together,  of  belonging  to 
something  larger  than  oneself,  and  devotion  to  teammates.  For 
example,  soldier  training  attempts  to  develop  a  cohesiveness  among 
members  of  units  so  that  when  they  are  faced  with  adverse 
situations  they  will  work  as  a  single  entity.  The  degree  to  which 
membership  awareness  affects  teams  varies  among  different  types  of 
jobs. 

148  To  what  degree  does  esprit  du  corps  affect  quality  of  team 
performance? 

149  To  what  degree  is  there  a  "culture"  of  team  membership,  e.g., 
uniforms,  physical  separation  of  team  members  from  others, 
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special  tokens,  special  treatment? 

150  How  explicit  are  expectations  for  team  obligations? 

151  How  explicit  are  expectations  for  team  benefits? 


TEAM  INTERDEPENDENCE 

This  section  of  the  survey  contains  questions  about  interdependence 
among  team  members.  Interdependence  in  a  team  is  the  degree  to 
which  the  overall  team  performance  depends  on  how  its  members 
perform  together.  For  example,  many  athletic  teams  (basketball, 
soccer,  etc.)  do  not  function  well  when  team  members  focus  on  their 
own  performance  and  ignore  the  performance  of  the  team  as  a 
whole.  By  contrast,  the  performance  of  a  track  or  a  gymnastics  team 
is  simply  the  sum  of  the  scores  earned  individually  by  each  of  its 
members. 

152  In  your  work,  to  what  degree  do  team  members  typically  adjust 
their  own  performance  and  blend  themselves  into  the  team  to 
get  the  best  overall  team  performance? 

153  What  proportion  of  team  members  are  typically  willing  to  accept 
interdependence? 

154  How  important  is  confidence  in  teammates? 

155  How  typical  is  confidence  in  teammates? 

156  To  what  degree  are  different  parts  of  the  task  mutually 
interdependent  while  they  are  being  performed?  For  example,  in 
a  musical  group  every  player  must  continually  synchronize  with 
all  the  others.  By  contrast,  in  some  factories  team  members  are 
not  aware  at  all  of  what  other  team  members  are  doing. 

157  To  what  degree  is  support  from  teammates  important  in 
generating  high  levels  of  motivation? 

158  To  what  extent  does  the  nature  of  the  task  encourage 
interdependence? 
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159  To  what  degree  does  the  work  culture  support  interdependence? 


TEAM  SHARED  SKILLS 

In  some  teams,  all  of  the  roles  that  members  can  take  are  Identical. 
Each  member  must  have  the  same  set  of  skills  as  every  other 
member.  In  other  teams,  each  member  has  a  very  different  role  that 
no  one  else  can  do,  and  the  skills  he/she  must  possess  may  be  quite 
different.  The  members  of  some  teams  may  have  such  different  skill 
requirements  for  their  respective  tasks  that  their  understanding  of 
the  work  is  completely  different.  For  example,  surgery  teams  have 
members  (e.g.,  anesthesiologist,  surgical  nurse,  surgeon,  attending 
physician)  whose  knowledge  and  skills  are  quite  different  and 
complementary.  The  questions  in  this  section  of  the  survey  ask  about 
the  skills  that  are  held  in  common  and  the  skills  that  are  different 
among  teammates. 

The  following  questions  are  not  about  degree  of  skill.  We  are 
concerned  here  only  with  whether  one  or  many  team  members  have 
a  particular  skill,  not  with  their  relative  competence  We  are  not 
interested  here  in  who's  the  best  or  worst  driver  on  the  team;  only, 
how  many  team  members  can  drive  the  truck? 

160  How  similar  are  teammates'  understanding  or  "mental  model"  of 
the  task? 

161  What  proportion  of  team  performance  skills  are  held  in  common 
by  all  team  members? 

162  How  important  is  it  for  team  members  to  know  what  their 
teammates  are  going  to  do? 

163  To  what  degree  do  teammates  share  or  trade  physical  tasks 
during  task  performance? 

164  How  common  is  active  mutual  encouragement  among  teammates 
during  task  performance? 
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165  To  what  degree  to  teammates  share  intellectual  tasks  during 
task  performance? 


TEAM  STABILITY 

This  section  of  the  survey  contains  questions  about  turnover  within 
teams  as  well  as  other  issues  of  stability.  Teams  vary  in  how  long 
they  are  composed  of  the  same  team  members.  Also,  they  vary  in 
how  often  the  roles  are  restructured  or  reorganized.  For  some  jobs, 
team  roles  must  be  loosely  organized  so  as  to  adapt  to  constantly 
changing  situations.  For  other  jobs,  a  single  organization  usually 
suffices.  Membership  stability  refers  to  issues  of  turnover  as  well  as 
frequent  changes  in  organization. 

166  How  long  do  teams  typically  stay  together? 

167  How  important  is  trust  in  other  team  members'  competence? 

168  How  common  is  it  for  teamwork  style  to  change  according  to  the 
situation?  For  example,  in  one  situation,  a  clear  and  commanding 
leader  may  be  needed;  in  another,  a  more  organic  group 
adaption  is  appropriate;  and  in  a  third  a  democratic  approach 
might  be  best. 

169  What  degree  of  autonomy  do  subteams  have  to  organize 
themselves  to  get  the  job  done? 
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SECTION  V:  ERRORS 


This  section  of  the  survey  addresses  errors  that  are  made  on  the  job. 
Different  types  of  errors  occur  in  different  jobs.  For  example,  some 
mistakes  are  physical  in  nature,  e.g.,  a  bolt  is  dropped  into  a  valve  a 
during  maintenance  task.  Other  errors  are  mental,- due  to  mistakes  of 
decision  or  judgment.  Each  of  the  four  subsections  of  this  component 
of  the  survey  corresponds  to  a  different  type  of  error. 


TYPICAL  ERRORS:  PHYSICAL  DIFFICULTY 

Physical  difficulty  in  tasks  often  result  in  physical  errors.  For  example, 
the  potential  for  error  increases  when  tasks  must  be  performed 
quickly,  with  great  strength,  or  with  great  precision.  This  section  of 
the  survey  contains  questions  about  the  physical  demands  of  your 
job  and  the  consequential  errors  that  result. 

We  are  interested  in  typical  errors.  There  may  be  some  very 
infrequent  errors  that  are  particularly  notable  because  they  have 
important  consequences,  but  in  his  section  we're  concerned  only  with 
the  typical  patterns  of  errors  that  are  most  commonly  seen. 

170  What  proportion  of  the  physical  parts  of  the  task  must  be 
performed  at  very  high  speed? 

171  What  proportion  of  the  physical  parts  of  the  task  must  be 
performed  with  very  high  precision? 

172  How  serious,  typically,  are  the  consequences  of  physical  errors? 

173  To  what  degree  can  physical  errors  be  repaired  or  compensated 
for? 


TYPICAL  ERRORS:  FATIGUE 

Fatigue  can  occur  from  lack  of  sleep,  high  levels  of  physical  exertion, 
low  to  medium  levels  of  physical  exertion  over  long  periods  of  time, 
mental  exertion,  or  emotional  stress.  Each  of  these  types  of  fatigue 
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has  the  potential  to  increase  the  likelihood  of  error  on  the  job.  The 
questions  in  this  section  address  issues  of  fatigue  and  error  that 
might  occur  in  your  job. 

174  What  kind(s)  of  fatigue  are  common?  Please  check  all  that  apply. 
General  physical 

Physical  overuse 

Emotional 

Mental 

Sleep  deprivation 
Cumulative  stress 

175  How  often  must  the  team  perform  the  task  when  fatigued? 

176  What  degree  of  fatigue  is  common? 

177  How  commonly  does  morale  fatigue  occur? 

TYPICAL  ERRORS:  MENTAL 

Mental  errors  can  be  distinguished  from  physical  errors  in  that  they 
are  largely  mistakes  in  judgment  or  decision.  Not  only  are  poor 
judgement  and  wrong  decisions  included  in  this  category,  but  errors 
of  anticipation,  of  failure  to  anticipate,  of  incorrect  or  inadequate 
assumptions,  or  of  misestimation  are  all  examples  of  the  range  of 
mental  error  of  interest.  The  questions  in  this  section  address 
decision  errors  that  might  occur  in  your  job.  Again,  it  is  the  typical 
errors  that  are  of  interest. 

178  How  common  are  decision  errors? 

179  How  avoidable  are  decision  errors? 

180  To  what  degree  is  there  a  clear  consensus  on  what  is  and  is  not 
an  error? 

181  To  what  degree  are  errors,  their  causes,  and  their  resolutions 
freely  discussed? 
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182  How  often  must  team  members  switch  among  subtasks  when 
performing  the  task? 

183  How  easy  is  it  to  discriminate  between  real  error  and  differences 
in  opinion  or  judgement? 

184  How  often  do  erroneous  beliefs  arise  and  take  root? 

185  How  easy  is  it  to  identify  patterns  of  common  errors?  ("Rookie" 
errors  might  be  an  example.) 


TYPICAL  ERRORS:  SELF-MANAGEMENT 

Efficient  and  effective  work  toward  a  goal  requires  self-management. 
Self-management  involves  knowing  when  progress  toward  the  goal 
has  been  achieved,  the  relative  contribution  of  that  progress,  the 
direction  of  the  progress,  and  what  other  work  must  still  be 
completed.  In  some  jobs,  it  is  possible  to  erroneously  believe  that  a 
solution  has  been  reached,  when  in  fact  the  problem  still  exists  and 
its  symptoms  are  only  being  disguised  or  masked.  For  example,  a 
physician  may  withdraw  his  or  her  attention  from  a  patient  when 
initial  signs  of  improvement  appear  and  fail  to  notice  the  indicators  of 
recurrence  that  would  have  been  obvious  if  the  right  amount  of 
attention  had  been  paid.  This  type  of  error  is  a  mistake  based  on  a 
poor  mental  allocation  of  attention,  that  is,  poor  self-management. 
The  two  questions  in  this  section  address  mistakes  in  self¬ 
management. 

186  How  frequently  is  a  short-term  "patch"  mistaken  for  real  long¬ 
term  task  completion? 

187  How  frequently  do  team  members  or  the  team  as  a  whole  focus 
so  much  on  a  current  subgoal  that  the  overall  goal  suffers? 
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SECTION  VI:  TASK  LEADERSHIP 

In  this  last  section,  we  ask  about  leadership.  For  some  kinds  of 
work,  leadership  is  an  essential  and  important  part  of  team 
performance.  For  other  kinds  of  work,  it  is  much  less  prominent.  In 
the  following  questions,  we  ask  about  the  role  of  leadership  in  your 
job.  As  with  all  of  the  other  sections  of  the  survey,  please  think  of 
job  performance,  rather  than  preparation,  training,  or  rehearsal. 

Task  leadership  is  an  important  component  of  many  jobs  that  require 
teamwork.  Leadership  qualities  include  motivational,  managerial, 
organizational,  and  technical  abilities,  as  well  as  others.  Leaders 
vary  in  how  well  they  do  their  job.  This  final  section  of  the  survey 
asks  you  about  leadership  in  your  work,  especially  as  it  is  relevant  to 
teamwork.  If  your  team  does  not  typically  have  a  leader,  do  not 
answer  the  following  questions.  Instead,  mark  each  question  "N/A" 
or  simply  skip  answering. 

188  How  thoroughly  does  a  team  leader  have  to  know  the  function  or 
utility  of  subordinate's  tasks,  that  is,  what  and  how  the  tasks 
contribute  to  the  overall  goal? 

189  How  thoroughly  does  a  team  leader  have  to  know  subordinate's 
tasks  with  respect  to  their  performance,  that  is,  how  to  actually 
do  the  task  himself  or  herself? 

190  How  often  do  team  members  typically  perceive  and  respect  their 
team  leader's  knowledge? 

191  How  often  are  team  leaders  perceived  as  being  willing  to  listen? 

192  How  often  do  team  leaders  themselves  display  good  team 
behaviors  (helping  out,  giving  and  receiving  feedback,  displaying 
attitudinal  interdependence,  etc.,  that  is,  leading  teamwork  skills 
by  example)? 

193  How  much  variability  is  there  among  team  leaders  in  approach  to 
feedback  and  guidance? 

194  How  easy  and  straightforward  is  it  to  judge  the  effectiveness  of 
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leadership? 

195  How  many  different  leadership  styles  are  used  for  the  task  (e.g., 
the  tough  style,  the  cheerleader  style,  the  lead-by-example  style, 
etc.)? 

196  How  common  is  micromanagement? 

197  How  aware  is  the  leader,  typically,  of  his  or  her  level  of 
performance? 

198  To  what  degree  is  there  direct  feedback  on  leadership 
effectiveness? 
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CRITIQUE  OF  THE  SURVEY 

A  few  words  about  the  intent  of  the  survey  and  the  assumptions 
underlying  it.  We  believe  that  teamwork  is  not  yet  well-understood, 
and  that  the  underlying  processes  need  to  be  better  illuminated.  Is 
the  teamwork  evidenced  by  a  fire  crew  the  same  as  the  teamwork 
displayed  by  a  jazz  quartet?  Are  the  differences  simply  a  matter  of 
different  degrees  of  emphasis  or  development  of  the  same  set  of 
underlying  psychological  components?  Or  are  different  components 
involved?  Unlike  more  specific  psychological  phenomena,  image 
recognition  for  example,  we  (as  a  research  community)  are  not  yet  in 
the  position  of  being  able  to  articulate  the  components  of  interest  in 
an  exact  and  exclusive  way,  and  to  make  precise  measurements  of 
them.  In  recognition  of  this,  the  survey  intentionally  sweeps  with  a 
broad  brush,  hoping  to  be  as  inclusive  as  possible.  The  goal  of  the 
attempted  inclusiveness  is  to  capture  what  is  common  between 
different  kinds  of  teamwork,  and  if  possible  to  identify  what  is  not 
common. 

Precise  surveys  are  only  possible  when  you  have  a  clear  statement 
or  decision  to  analyze,  and  when  you  meet  the  important  additional 
condition  that  you  have  a  denumerable  (and  manageably  small)  list  of 
alternatives  from  which  to  choose.  In  AI  jargon,  this  is  a  "structured 
selection"  problem.  You  can't  perform  a  structured  selection,  though, 
until  you've  bounded  the  space  and  enumerated  the  alternatives. 

With  respect  to  teamwork,  the  training  research  community  has  not 
yet  done  that.  Studying  training  is  much  more  what's  called  a 
"constructive"  problem,  i.e.,  a  process  of  taking  candidate 
components  or  ideas,  trying  out  different  combinations  and 
arrangements,  and  exploring  the  space  of  possibilities  with  the 
expectation  that  an  increasingly  focused  picture  will  (eventually) 
emerge.  This  is  a  bit  more  like  art  than  science,  or  put  another  way, 
more  like  design  than  like  implementation.  Surveys  are  not  nearly  as 
apt  for  constructive  problems  as  they  are  for  structured  selection 
(unless  you  allow  essay  questions,  which  pretty  much  obliterates 
precision).  Our  goal  with  the  survey  is  to  provide  data  points  useful 
to  bounding  the  problem  space  of  teamwork  and  contributing  toward 
understanding  its  internal  structure. 
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There  can  be  no  doubt  that,  ideally,  we  want  our  questions  and 
stimuli  to  be  as  specific  as  possible,  so  that  each  respondent  has  an 
identical  understanding  of  the  meaning  of  each  question.  Otherwise, 
a  single  question  can  end  up  being  different  questions  for  different 
respondents.  Then  how  could  we  make  sense  of  the  results? 
Empirically,  though,  we  ran  sixty  people  and  none  of  them  had 
trouble  answering  the  questions.  So  the  evidence  is  suggestive  that 
the  questions  resolve  to  single,  strongly  preferred  interpretations. 

Most  surveys  are  more  precise  than  the  one  we've  created.  The 
problem  is  not  that  we're  sloppy,  unscientific  dummies.  The  problem  is 
in  what  we're  trying  to  do.  My  belief  is  that  imposing  clear,  crisp 
distinctions  on  teamwork  at  this  point  would  be  artificial:  It  would 
assume  more  knowledge  than  yet  exists,  and  would  therefore  be  bad 
science. 

The  most  serious  criticism  of  the  survey  was  offered  by  Angelo 
Mirabella,  Ph.D.:  "Inadequate  effort  to  insure  that  the  respondent 
establishes  and  maintains  a  well-defined,  consistent  frame  of 
reference."  This  is  a  powerful  critique.  The  problem  is,  some  tasks 
have  well-defined  categories  of  episode  that  can  productively  be 
adopted  as  a  reference  point  in  answering  the  questions,  and  some 
don't.  For  example,  the  "incident  command  of  a  multi-alarm  fire"  is  a 
clear  subcategory  of  fire  command.  But  for  an  emergency  room 
physician,  so  many  such  categories  exist,  each  with  so  many 
variations,  that  to  constrain  the  questions  to  a  single  incident  or 
category  of  incidents  would  be  to  arbitrarily  and,  to  my  mind,  not  very 
sensibly  restrict  the  doctor's  freedom  to  describe  her  work.  Worse,  it 
would  give  a  false  reading  of  what  she  actually  does— Her  wide  range 
of  tasks  is  an  essential  element  of  her  work. 

Even  within  a  nicely  defined  task,  competitive  basketball,  for  instance, 
such  strong  frames  of  reference  might  or  might  not  be  useful.  Using 
the  basketball  example,  risk  acceptance  may  be  significantly  different 
in  the  fourth  quarter  than  the  first,  if  your  team  is  ahead  or  behind 
(and  by  how  much),  if  it's  the  regular  season  or  the  playoffs,  if  it's  the 
first  game  of  the  playoffs  or  the  last,  and  so  on. 

The  frame  of  reference  problem  came  up  over  and  over  again  as  we 
designed,  and  redesigned,  and  argued  about  the  survey.  Do  we 
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restrict  to  a  single  incident,  or  to  a  single  category  of  incident,  or  do 
we  try  to  make  our  intentions  clear  and  leave  it  to  the  respondents 
to  decide  whether  one  event  or  category  of  events,  or  the  entirety  of 
their  duties  should  be  the  base  from  which  they  draw  in  answering 
questions? 

We  chose  the  more  general,  less  restrictive  approach.  I  believe  that 
as  a  research  community,  we  are  still  in  the  process  of  "successive 
approximation"  in  studying  teamwork.  Methodologically,  it  would  be 
much  nicer  to  have  crisp  and  unambiguous  boundaries.  But  because 
we  do  not  yet  know  enough  about  teamwork  to  impose  strict 
boundaries  without  risking  a  self-validating  survey,  we  chose  to  hew 
to  the  generalist  approach. 

One  further  factor  pushed  us  in  the  direction  of  leaving  it  up  to  the 
respondent  to  decide.  We  were  aware  that  in  some  kinds  of 
teamwork,  an  intrinsic  part  of  the  job  is  to  perform  multiple,  disparate 
subtasks  concurrently,  and  to  dynamically  negotiate  and  trade 
responsibilities  for  these  subtasks  among  team  members.  Our  ER 
doctor  is  one  example  of  this  sort  of  task;  nursing  is  another. 

(Because  competence  in  such  load-balancing  and  load-sharing  is  a 
critical  element  of  effective  teamwork  in  nursing,  it  is  highly  germane 
to  nursing  training.)  We  did  not  want  to  artificially  limit  respondents 
with  these  sorts  of  multitasking  kinds  of  work. 

The  establishment  of  a  firm  and  clear  frame  of  reference,  and  also, 
more  importantly,  providing  a  survey  instrument  that  can  be  sensibly 
used  by  multiple  populations  runs  into  the  problem  of  the  degree  to 
which  one  size  can  fit  all.This  is  called  in  the  AI  world  the  "strong 
methods/weak  methods"  problem,  and  has  been  discussed  since  the 
50's  (an  outgrowth  of  seeing  the  limits  of  Newell  &  Simon's  General 
Problem  Solver). 

A  "strong  method"  is  one  that  is  completely  specific  to  a  class  of 
problem  and  is  known  to  generate  an  accurate  answer.  A  "weak 
method"  is  one  that  is  very  general  and  can  be  applied  over  a  range 
of  problems.  If  you  have  a  linear  optimization  problem,  linear 
programming  is  a  strong  method,  and  of  course  is  to  be  preferred.  On 
the  other  hand,  if  you  have  a  medical  prescriptive  problem,  strong 
methods  do  not  exist,  so  the  issue  is  which  weak  methods  you  apply, 
when  and  how. 
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The  more  tightly  you  can  focus  a  problem,  the  more  likely  you  are  to 
find  a  strong  method.  On  the  other  hand,  the  more  tightly  you  focus  a 
problem,  the  narrower  your  view,  and  the  less  broadly  useful  are 
either  your  methods  or  your  results.  Since  we  wish  for  broad 
coverage,  we  wrote  questions  amenable  to  a  broad  range  of 
answers. 

While  strong  methods  might  seem  desirable,  had  we  in  fact  used  a 
much  more  tightly  focused  approach  in  the  survey,  we'd  be  guilty  of 
pre-fitting  what  we  wish  to  study.  Since  we  want  to  see  what  kinds 
of  patterns  of  teamwork  emerge,  a  tight  focus  would  have  forced  us 
to  make  a  number  of  strong  assumptions  about  what  we  were  going 
to  see  prior  to  asking  our  respondents.  You  only  get  a  strong  focus  if 
you  have  a  strong  framework.  I've  expressed  my  belief  that  the 
framework  for  studying  teams  is  immature.  Therefore,  it  seems  to  me 
it  would  be  difficult  to  narrow  the  focus  without  biasing  the  survey 
toward  our  own  preconceptions.  To  avoid  this,  we  had  to  write  a 
more  general  instrument. 

As  teamwork  becomes  better  understood  and  more  empirical  data  is 
gathered,  stronger,  more  focused  research  will  be  possible.  Based  on 
our  reading,  though,  teamwork  is  not  well-understood  yet,  and 
therefore  a  more  narrowly  focused  survey  would  run  the  risk  of  being 
self-validating. 

To  bring  this  back  to  the  specific  question,  whether  to  answer  as  an 
individual  team  member  or  on  behalf  of  the  team  as  a  whole,  the 
question  is,  how  different  are  the  answers  between  the  two 
perspectives? 

Many  questions  are  unaffected  by  the  distinction,  i.e.,  questions 
about  the  nature  of  the  task  and  processes  ("Once  a  diagnosis  has 
been  made,  how  many  actions  can  be  prescribed?")  Many  more 
questions  simply  require  one  or  the  other  perspective  ("To  what 
degree  does  esprit  du  corps  affect  quality...?";  "How  motivated, 
typically,  are  team  members...?") 

We  are  left  with  circumstantial  and  outcome  questions  ("To  what 
extent  is  the  physical  environment  in  which  the  task  is  performed 
fast-changing?";  "How  long  after  a  solution  is  implemented  is  it 
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apparent  that  it  was  correct  or  incorrect?")  When  there  are  significant 
circumstantial  or  outcome  factors  that  vary  widely  between  team 
members,  then  the  answers  may  differ  between  the  individual  and 
the  team  perspective. 

We  don't  think  the  difference  is  significant.  We  rely  on  two 
interrelated  assumptions:  (a)  coherent  team  binding;  and,  (b) 
respondent  representativeness.  In  coherently  bound  teams,  the 
circumstantial  and  outcome  factors  to  which  the  distinction  applies 
will  be  perceived  as  shared.  It's  the  team  that  takes  the  floor,  the 
team  that  wins  or  loses.  This  is  nearly  definitional:  If  circumstantial 
factors  are  not  perceived  as  shared,  the  team  will  not  experience 
itself  as  coherently  bound.  (Do  employees  of  large  companies  feel 
they  are  on  the  same  "team"  as  their  CEO?)  We  also  assume 
respondent  representativeness,  that  the  person  completing  the 
survey  is  a  "regular"  team  member.  We  intended  this  when  we 
developed  our  definition  of  team  membership,  which  was  constructed 
to  explicitly  exclude  individualistic  interactions  and,  in  general, 
"external"  team  affiliates.  That  a  person  or  role  is  necessary  to  team 
functioning  does  not  mean  that  person  or  role  is  directly  part  of  the 
team.  Coaches  and  commanders  are  explicitly  excluded,  unless  they 
actually  participate  with  the  team  in  the  field.  (They  can  be  part  of 
coaching  or  command  teams,  of  course.) 

In  one  very  specific  criticism,  the  question,  "How  much  do  solutions 
vary  in  quality?"  is  thought  to  be  ambiguous  with  respect  to  (a) 
variation  in  solution  quality  between  alternative  solutions  to  a  single 
problem  or  episode,  or  (b)  variation  of  solutions  for  a  category  of 
problem. 

We  believe  these  alternatives  are  less  different  than  they  appear  to 
be  on  the  surface.  For  a  mathematical  equation,  there  typically  are 
either  only  one  or  a  small  number  of  correct  solutions.  (There  are 
multiple  solutions  to  Einstein's  equations  for  General  Relativity,  for 
example.)  Because  this  is  true  for  all  mathematics,  distinguishing 
between  a  particular  problem  and  a  general  class  of  problem  is  not 
especially  useful,  because  the  "quality  of  solution"  answer  is  the 
same  either  way.  At  the  other  end  of  the  spectrum,  for  some  kinds  of 
design  problems,  the  solution  space  is  huge,  and  there  are  always 
wide  variations  in  solution  quality.  So  here  as  well  distinguishing  a 
specific  problem  and  a  class  of  problem  doesn't  generate  significant 
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useful  information,  because  again  the  answer  is  the  same  either  way. 

What  about  intermediate  problems,  like  medical  diagnosis  or  military 
situation  assessment?  When  we  consider  solution  quality  in  these 
cases,  we  are  probably  really  considering  uncertainty.  (On  the 
assumption  of  the  competent  practitioner:  The  population  of 
competent  practitioners  would  quickly  converge  on  a  small  number  of 
interpretations,  but  would  vary  in  their  solutions  according  to  the 
uncertainty  of  the  situation.)  But  since  throughout  the  survey  we  ask 
and  encourage  the  respondents  to  discuss  how  things  usually  are, 
and  to  skip  the  boundary  cases  except  when  we  explicitly  ask  for 
them,  then  the  answer  for  a  typical  specific  situation  (with  typical 
uncertainty)  should  map  isomorphically  to  the  answer  for  the  class  of 
situation.  Again,  we  end  up  with  the  same  answer. 

I  believe  that  we  unconsciously  include  the  nature  of  solutions  as  a 
priori  conditions  for  our  notions  of  what  comprises  a  task,  i.e.,  the 
perceived  nature  of  solutions  is  part  of  the  task  bounding  process. 

We  don't  classify  a  medical  question  like  "do  implants  cause 
connective  tissue  damage?"  with  one  like  "how  long  should  life  be 
prolonged  in  extreme  cases?"  Though  they  both  refer  to  medical 
issues,  the  kinds  of  solutions  they  have  are  so  different  we 
automatically  put  them  in  different  conceptual  bins  (epidemiology  and 
medical  ethics,  respectively).  If  an  epidemiologist  takes  the  survey, 
we  would  expect  him  or  her  to  answer  questions  with  respect  to 
epidemiology,  not  medical  ethics. 

My  view  is  that  the  study  of  teamwork  is  not  mature  and  that  we 
have  no  choice  but  to  deal  with  many  concepts  that  are,  at  this  point 
at  least,  ontologically  vague.  Perhaps  my  view  is  too  heavily 
influenced  by  my  background,  but  it  is  difficult  for  me  to  accept  that 
the  precision  that  exists  in  computer  science  concepts  like  "efficiency 
of  queuing  mechanisms"  or  "optimal  search  of  balanced  binary  trees" 
can  be  found  in  teamwork  concepts  like  "monitoring",  "vigilance",  or 
"reciprocal  backup."  Some  psychological  concepts  are  elegantly 
precise,  but  it  doesn't  appear  to  me  we're  there  yet  with  teamwork. 
Because  there  is  a  far  larger  range  of  subject  matter  crammed  into 
our  current  conceptual  vocabulary  for  teamwork  than  there  is  for 
balanced  b-trees,  we  are  less  ready  to  impose  the  precision  available 
in  logic  or  programming  to  the  psychological  objects  of  our  study. 
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The  movement  over  the  past  twenty  years  in  making  psychology 
more  precise  is  impressive,  and  my  expectation  is  that  the  coming 
twenty  years  will  see  even  more  progress,  especially  as  we  integrate 
neurophysiology  and  artificial  intelligence  results  into  the  field.  When 
we  have  reached  that  degree  of  precision  in  the  study  of  teamwork, 
then  the  generalist  approach  we  have  adopted  in  the  survey  will  be 
obsolete.  But  it  is  my  belief  that  we  have  not  yet  reached  that  state. 
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RESPONDENTS 


Sixty  subjects  participated,  in  the  following  categories: 

Athletics  (19) 

Engineering  (3) 

ER  medicine  (MDs)  (3) 

Firefighting  (6) 

Improvisational  music  (3) 

Marines  (6) 

Nursing  (6) 

Police  (3) 

Teaching  (team)  (3) 

Writing/editing  (3) 

Other  (5) 

We  recruited  a  minimum  of  three  subjects  in  each  area  in  order  to  be 
able  to  check  intersubject  reliability. 
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INSTRUCTIONS  GIVEN  TO  RESPONDENTS 


Following,  verbatim,  are  the  instructions  given  our  subjects. 

We— the  authors  of  the  survey— are  trying  to  better  understand 
teamwork.  We  consider  "teamwork"  to  be  any  activity  where  (a) 
people  work  together  toward  a  common  goal;  (b)  members  know 
that  they  are  members  of  a  team  and  know  who  their  teammates 
are;  and  (c)  team  members  must  interact  during  the  work  in  order  to 
achieve  the  goal.  Your  answers  to  the  following  survey  questions  will 
help  us  understand  the  nature  of  teamwork  in  your  area  of  expertise. 

The  questions  are  grouped  into  seven  categories:  conditions,  the 
structure  of  your  task,  decision-making,  knowledge,  specific 
teamwork  issues,  typical  errors,  and  the  role  of  leadership. 

Examples  of  the  kinds  of  teams  we're  interested  in  range  pretty 
wide:  sports  teams,  firefighters,  military  units,  jazz  musicians, 
software  project  groups,  emergency  room  teams,  and  many  many 
others. 

The  only  restriction  we  place  on  whether  someone  should  be 
considered  part  of  the  team  is  that  interactions  between  the  person 
in  question  and  the  rest  of  the  team  go  both  ways.  A  coach  would 
not  be  considered  part  of  an  athletic  team.  Nor  would  a  symphony 
conductor  be  considered  part  of  the  orchestra,  nor  a  Battalion 
Commander  part  of  the  battalion  team.  The  Battalion  Commander 
and  his  or  her  staff  would  be  a  team,  though,  as  would  a  conductor 
and  a  composer,  or  a  player-coach  of  an  athletic  team.  The 
distinguishing  feature  is  the  interplay  between  team  members.  If  it 
goes  in  both  directions,  then  the  person  is  (by  the  definition  we  use 
in  this  survey)  a  team  member.  If  it  only  goes  one  way,  e.g.,  as  in 
issuing  orders  or  directions,  then  even  though  the  person  may  be 
necessary  to  the  team's  functioning,  he  or  she  is  not  considered  a 
team  member. 

Taking  the  survey  is  simple.  You  simply  circle  the  number  that  seems 
right  to  you  on  each  scale.  Most  questions  only  want  one  selection, 
so  if  you  make  a  mistake  or  change  your  mind,  just  cross  our  your 
original  answer  and  circle  the  number  you  now  think  is  most  accurate. 
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If  a  question  doesn't  really  pertain  to  your  work,  you  can  always 
circle  "Not  Applicable"  ("N/A").  Please  answer  only  the  questions  that 
make  sense  to  you.  You  can  always  pass  a  question  by;  you  never 
have  to  answer  a  particular  question. 

Each  question  is  presented  with  two  "anchor  points,"  written  below 
the  scale.  These  can  be  thought  of  as  the  boundaries  for  the  range  of 
answers  to  the  particular  question.  We've  typically  made  the  ends  of 
the  scales  pretty  strong  statements.  That's  so  if  something  is  really 
true  about  your  work,  you  can  tell  us.  Don't  be  shy  about  making 
strong  statements.  At  the  same  time,  try  to  be  realistic  in  your 
answers. 
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HYPOTHESES 


We  generated  and  tested  three  hypotheses  about  relationships 
between  task  elements  and  teamwork  and  leadership. 

Teamwork 

Hypothesis  1. 

Stressful  and/or  dangerous  environments  are  strongly  correlated 
with  high  scores  on  the  teamwork  issues  of  mutual  monitoring, 
reciprocal  backup,  and  esprit  du  corps. 

Mutual  monitoring  is  highly  correlated  with  a  physically  stressful 
environment,  r  =  .556,  p  <  .0001. 

Reciprocal  backup  is  highly  correlated  with  a  physically  stressful 
environment,  r  =  .538,  p  <  .0001. 

Esprit  du  corps  is  highly  correlated  with  a  physically  stressful 
environment,  r  =  .558,  p  <  .0001. 

Mutual  monitoring  is  highly  correlated  with  a  physically  dangerous 
environment,  r  =  .345  ,  p  <  .01. 

Reciprocal  backup  is  highly  correlated  with  a  physically  dangerous 
environment,  r  =  .538,  p  <  .0001. 

Esprit  du  corps  is  highly  correlated  with  a  physically  dangerous 
environment,  r  =  .521,  p  <  .0001. 

Leadership 

Hypothesis  2. 

Seriousness  of  outcome  is  strongly  correlated  with  respect  for  a  team 
leader's  knowledge  and  with  his  or  her  modeling  of  teamwork 
behaviors. 

Seriousness  of  outcome  does  not  appear  to  be  correlated  with  a 
team  leader's  knowledge,  r  =  .189,  p  >  .1. 

Seriousness  of  outcome  does  not  appear  to  be  correlated  with  a 
team  leader's  modeling  of  teamwork  behaviors,  r  =  .06,  p  >  .1. 
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Hypothesis  3. 

Degree  of  uncertainty  in  the  task  environment  is  only  weakly 
correlated  with  respect  for  a  team  leader's  knowledge  and  with  his  or 
her  modeling  of  teamwork  behaviors.  It  is  slightly  more  strongly 
correlated  with  the  degree  to  which  the  team  leader  is  perceived  to 
be  self-aware. 

Degree  of  uncertainty  in  the  task  environment  is  correlated  with  a 
team  leader's  knowledge,  r  =  .353,  p  <  .01. 

Degree  of  uncertainty  in  the  task  environment  is  correlated  with  a 
team  leader's  modeling  of  teamwork  behaviors,  r  =  .270,  p  <  .05. 

Degree  of  uncertainty  in  the  task  environment  is  correlated  with  the 
degree  to  which  the  team  leader  is  perceived  to  be  self-aware,  r  = 
.385,  p  <  .005. 
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DISCUSSION 


Hypothesis  1  claimed  "Stressful  and/or  dangerous  environments  are 
strongly  correlated  with  high  scores  on  the  teamwork  issues  of 
mutual  monitoring,  reciprocal  backup,  and  esprit  du  corps." 

Confirmation  of  this  hypothesis  is  unsurprising  -  in  tough  and/or 
hostile  environments,  teammates  stick  together.  With  respect  to 
mutual  monitoring  the  evenness  of  the  very  strong  correlation  is 
mildly  surprising  in  that  high  rankings  were  also  given  to  that  variable 
(mutual  monitoring)  by  the  improvising  musicians,  who  did  not  rate 
their  environments  as  particularly  dangerous. 

Hypothesis  2  claimed  "Seriousness  of  outcome  is  strongly  correlated 
with  respect  for  a  team  leader's  knowledge  and  with  his  or  her 
modeling  of  teamwork  behaviors." 

Outcome  seriousness  did  not  correlate  with  either  respect  for  a  team 
leader's  knowledge  ar  modeling  of  teamwork  behaviors.  This  is  a 
surprise.  We  had  two  subgroups  who  routinely  deal  with  serious 
outcomes,  doctors  and  nurses,  and  Marines,  police,  and  firefighters. 
The  two  groups  are  very  different  with  respect  to  leadership.  Our  two 
medical  groups  tend  to  be  fairly  (nurses)  to  highly  (ER  doctors) 
autonomous  and  collegial,  while  Marines,  police,  and  firefighters  are 
hierarchical  command  organizations.  In  a  hierarchical  command 
organization,  leaders  are  not  supposed  to  model  teamwork 
behaviors;  rather,  they  have  distinct  responsibilities  which  tend  to  be 
other  than  those  of  their  subordinates.  Thus,  in  retrospect,  perhaps 
we  should  not  have  expected  the  command  organizations  to  show 
the  predicted  correlation. 

That  doesn't  explain  the  lack  of  correlation  with  respect  to  leader's 
perceived  knowledge,  though.  A  persuasive  explanation  is  not  ready 
to  hand. 

Hypothesis  3  claimed  "Degree  of  uncertainty  in  the  task  environment 
is  only  weakly  correlated  with  respect  for  a  team  leader's  knowledge 
and  with  his  or  her  modeling  of  teamwork  behaviors.  It  is  slightly 
more  strongly  correlated  with  the  degree  to  which  the  team  leader  is 
perceived  to  be  self-aware." 
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We  didn't  expect  to  find  correlations  on  the  first  two  parts  of  the 
hypothesis,  but  we  did.  The  correlation  is  technically  higher  for  the 
third  part,  but  not  substantively. 

Our  reasoning  in  proposing  the  hypothesis  was  that  external  factors 
(degree  of  uncertainty  outside  the  control  of  the  practitioner)  should 
show  little  correlation  with  team  members  views  of  their  leaders. 

Two  explanations  of  our  findings  suggest  themselves:  Selection  and 
attribution.  Selection  would  occur  if  the  uncertainty  of  the 
environment  had  the  effect  of  making  the  selection  of  leaders  more 
rigorous,  such  that  those  selected  for  leadership  are  genuinely 
deserving  of  their  team's  respect.  Attribution  would  occur  if  the 
psychological  pressure  of  uncertainty  predisposed  team  members  to 
attribute  competence  to  their  leaders,  independent  of  evidence, 
simply  as  a  way  to  decrease  that  pressure  by  forming  the  belief  that 
they  are  in  "good  hands." 

One  potential  measure  to  test  this  would  be  to  develop  some  means 
of  assessing  volatility  in  assessment  of  leadership.  If  respect  is 
genuine,  then  volatility  should  remain  low  despite  temporary  highs 
and  lows  in  team  performance.  On  the  other  hand,  if  it  is  a  projection, 
we  would  expect  a  higher  likelihood  of  abrupt  reversal  if  team 
members  felt  "let  down"  by  the  leader,  even  if  variations  in  team 
performance  had  nothing  to  do  with  his  or  her  performance. 
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HOW  TO  FIGURE  OUT  WHAT  YOUR  TEAM  IS 
REALLY  DOING 

People  have  been  trying  to  figure  out  what  is  really  going  on 
for  a  long  time.  (Some  have  been  more  successful  than  others.) 
Psychologists  have  developed  a  number  of  task  analysis 
methods  in  order  that  highly  specific  descriptions  of  jobs  can  be 
written. 

In  this  section,  we  relate  task  analysis  and  cognitive  task 
analysis  techniques.  First,  we  discuss  task  and  cognitive  task 
analysis  for  individuals,  then  for  groups.  Before  doing  that, 
though,  some  further  orientation  might  be  useful.  We'll  provide 
some  general  categories  of  tasks,  then  a  progression  of  ideas 
about  how  tasks  and  their  environments  interact. 
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SOME  VERY  GENERAL  KINDS  OF  TASKS 

What  do  people  do?  There  are  many  ways  to  group  and 
differentiate  kinds  of  work,  and  many  taxonomies  representing 
different  perspectives.  An  approach  that  has  been  popular  in 
cognitive  psychology  and  artificial  intelligence  categorizes 
human  tasks  as  either  convergent  or  divergent. 

Convergent  tasks  have  ("converge  on")  one  right  answer. 
Manipulation,  monitoring,  identification,  interpretation, 
diagnosis,  and  debugging  are  typically  viewed  as  convergent. 

Divergent  tasks,  on  the  other  hand,  may  have  many  possible 
satisfactory  answers.  Prescription,  design  (which  can  be  viewed 
as  arranging  objects  under  constraints),  planning  (similar  to 
design,  but  arranging  actions  rather  than  objects  under 
constraints),  and  rationalizing  (explaining,  describing, 
motivating,  etc.)  are  all  typically  seen  as  divergent. 

Generally,  other  common  tasks,  e.g.,  control,  repair, 
configuration,  scheduling,  prediction,  instruction,  and 
evaluation,  can  be  composed  from  the  more  clearly  defined 
"basic"  tasks  listed  above.  As  a  rough  heuristic,  if  the  truth  is  in 
the  world,  the  problem  is  probably  convergent;  if  truth  is  in  the 
mind,  it's  probably  divergent. 
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TASKS-IN-PLACE:  DISTRIBUTED 
REPRESENTATION 

Interest  in  distributed  representation  comes  from  the  insight 
that  complex  behavior  can  be  accounted  for  by  relatively  simple 
mechanisms  in  a  complex  environment  (Simon  1981;  Brooks 
1992),  and  the  empirical  fact  that  different  representations  of 
the  same  problem  can  have  dramatic  effects  on  problem-solving 
performance  (Hayes  &  Simon  1977). 

"Distributed  representation"  happens  in  problem  solving  when 
part  of  the  way  the  problem  is  represented  is  internal  to  the 
problem-solver,  and  part  is  eternal  in  the  environment.  Two 
mathematicians  solving  a  problem  at  a  blackboard  is  an 
example.  From  this  view,  an  external  representation  is  not  just 
a  peripheral  aid  to  cognition,  but  a  necessary  component  of  a 
distributed  cognitive  task.  Zhang  and  Norman  (1994)  found  five 
properties  of  external  representation,  as  relevant  to  distributed 
representations: 

1.  External  representations  provide  memory  aids 

2.  External  representations  can  provide  information  that  can 
be  directly  perceived  and  used  without  being  interpreted 
and  formulated  explicitly.  This  is  consistent  with  Gibsonian 
affordances  (Gibson  1979). 

3.  External  representations  can  anchor  and  structure 
cognitive  behavior,  especially  to  the  extent  that  the 
external  representation  is  instantiated  in  physical 
structures  and  artifacts. 

4.  External  representations  can  change  the  nature  of  the  task 
as  seen  from  the  perspective  of  the  problem  solver,  i.e., 
appropriate  externalization  of  representation  often  makes 
tasks  easier. 

5.  External  representations  are  an  indispensable  part  of  the 
representational  system  of  any  distributed  cognitive  task. 

In  the  past,  task  analyses  took  external  representations  to  a 
greater  or  lesser  extent  for  granted.  Thoughtful  examination  of 
the  distribution  of  representation  is  deserving  of  a  prominent 
place  in  all  kinds  of  task  analyses. 
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TASKS-IN-PLACE:  SITUATED  ACTION 


"Situated  action"  is  the  claim  the  cognition  cannot  be 
understood  as  a  "thing"  independent  of  the  context  in  which 
cognition  takes  place.  There  is  some  persuasive  evidence 
supporting  this  view: 

•  Independent  Ericcson/Simon  &  Johnson-Laird  results  show 
that  how  hard  a  problem  is  often  centers  not  on  stack 
depth  (how  many  things  you  have  to  think  about)  but  on 
representational  transformations  (how  many  ways  you 
have  to  be  able  to  look  at  the  problem).  Changes  in 
representation  are  closely  tied  to  where  and  with  what  the 
problem  is  presented. 

•  Hutchins'  work  shows  distribution  of  "cognition"  and 
representation  between  people  and  their  environment,  and 
even  more,  that  intelligent  behavior  cannot  be  accounted 
for  (that  is,  is  underspecified)  except  by  looking  at  the 
human/environment  system  as  a  whole. 

•  Edelman  proposes  that  the  essence  of  intelligence  is  not  in 
a  applying,  but  in  selecting  (distributed)  representation. 

There  is  a  lot  more,  all  pointing  to  the  essential  nature  of 
action-embedded-in-the-world,  that  the  "inside  the 
head/outside  in  the  world"  dichotomy  is  often  misleading  or 
even  false.  Situated  action  researchers  claim  that  high-level 
human  conceptual  abstractions  and  extremely  low-level 
material  instantiations  interact  to  produce  behavior. 

It  would  be  hard  to  make  a  more  commonsensical  statement. 
Removed  from  academic  language,  all  it  says  is,  people  live  in 
the  world,  and  that's  where  they  do  their  thinking.  For  task 
analyses,  it  makes  sense  to  direct  attention  to  how  the 
situation  and  the  desired  action  interact,  and  how  changes  in 
the  situation  trigger  new  actions. 
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TASKS-IN-PLACE:  ACTIVITY  THEORY 

Activity  theory  is  the  strongest  expression  of  situated  action. 

The  two  core  notions  of  activity  theory  are: 

•  Mediation.  "All  human  experience  is  shaped  by  the  tools 
and  sign  systems  we  use.  Mediators  connect  us  organically 
and  intimately  to  the  world;  they  are  not  mere  filters  or 
channels  through  which  experience  is  carried."  (Nardi  1996) 

.  Consciousness.  "Consciousness"  is  defined  in  Vygotsky's 
sense,  that  is,  the  unification  of  attention,  intention, 
memory,  reasoning,  and  language. 

This  is  definitely  a  place  where  the  specific  methodological 
application  of  the  concepts  is  critical,  since  people  seem  to  feel 
fine  about  using  the  word  "conscious"  to  stand  for  anything 
they  want  it  to.  (Exercise  for  the  reader:  List  anything  people 
do  that  can  t  be  explained  by  reference  to  one  or  more  of  the 
attributes  in  Vygotsky's  list.) 

The  thrust,  then,  of  the  methodological  argument  is  that 
behaviors,  both  individual  and  especially  group,  are  not  just 
"embedded"  in  some  physical  artifact  context.  They  are 
essentially  intertwined.  For  that  reason,  the  appropriate  unit 
for  analysis  is  the  whole  system,  and  decomposition  should 
begin  there.  This  is  in  contrast  with  traditional  task  analysis, 
which  starts  with  the  activities  of  each  individual. 

Not  that  activity  theory  minimizes  the  importance  of  the 
individual;  according  to  Kaptelinin,  activity  theory  insists  on  an 
essential  asymmetry,  focusing  on  the  importance  of  motive  and 
consciousness.  Artifacts  are  mediators  of  human  thought  and 
behavior.  This  is  slightly  different  than  Hutchin's  systematicity, 
but  both  clearly  differ  from  both  behavioral  and  cognitive 
psychology. 

Nardi,  Hutchins,  and  others  claim  this  view  yields  analytic  power 
not  attainable  by  traditional  task  analysis.  Task  analysis  is  post 
facto,  following  practice  (though  it  can  lead  to  changes  in 
practice).  Currently,  most  artifact  design,  and  training  design  for 
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that  matter,  is  also  in  a  sense  post  facto— it  follows  practice.  In 
fact,  we  look  at  successful  artifacts  or  training  programs  to 
understand  why  they  work.  If  activity  theory  provides  additional 
analytic  or  (especially)  predictive  power,  then  it  is  useful. 

Task  analysis  and  activity  theory  are  complimentary.  Activity 
theory  that  doesn't  "bottom  out"  with  specific,  concrete  actions 
is  too  vague  to  use.  On  the  other  hand,  task  analysis  that 
doesn't  understand  the  system  of  which  the  behavior  of 
interest  is  an  interwoven  part  relies  on  luck  to  make  sense. 

That  may  seem  strong  -  particularly  since  we're  about  to 
describe  task  analysis  in  quite  a  lot  of  detail  -  but  it's  true  to  the 
extent  that  task  analysis  works  best  when  the  tasks  chosen 
are  sufficiently  "compartmentalized"  to  minimize  interactions 
with  task  context,  and  sufficiently  non-symbolic  to  minimize 
cognitive  implications  of  task  steps.  If  either  of  the  conditions  is 
violated,  task  analysis  is  less  effective.  Stated  in  different 
words,  the  greater  the  role  of  tacit  knowledge  and  behavioral 
fluendy,  the  less  effective  is  task  analysis.  Since  the  focus  of 
task  analysis  is  on  articulating  specific  sets  of  well-defined 
steps  that  lead  to  a  task  goal,  the  hard  part  of  task  analysis, 
defining  and  selecting  tasks  to  analyze,  is  left  outside  the 
method.  Since  activity  theory  seems  to  address  this  kind  of 
issue,  it  is  worth  consideration  for  training  design. 
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WHAT  NEEDS  TO  BE  UNDERSTOOD? 

In  the  following  sections,  we  distinguish  behavior  and  cognition. 
As  should  be  apparent,  we  believe  this  is  a  "distinction  of 
convenience;"  you  don't  really  get  one  without  the  other. 
However,  it  is  very  useful  fiction. 

We  will  refer  to  behavior  as  what  can  be  directly  observed. 
Behavior  occurs  in  steps,  and  all  the  steps  can  be  observed  and 
can  reasonably  be  said  to  have  a  beginning,  a  middle,  and  an 
end.  Changing  a  tire  is  a  clear  example  of  a  behavior. 

To  a  great  extent,  the  structure  of  the  task  in  the  world 
determines  the  degree  of  behavioral  binding.  The  nature  and 
sequence  of  steps  in  changing  a  tire  are  almost  entirely 
constrained  by  how  wheels  are  attached  to  cars,  and  what 
wheels  and  cars  are. 

For  traditional  task  analysis  -  which  is  very  much  behaviorally 
oriented  -  the  best  kinds  of  tasks  are  those  for  which  there  is 
only  one  way  to  do  each  step,  only  one  way  to  sequence  the 
steps,  and  it  is  always  clear  when  the  task  is  completed. 

Always  bearing  in  mind  that  the  distinction  is  artificial,  we'll  call 
"cognitive"  any  task  (or  portion  of  a  task)  with  non-observable 
steps.  Reading  a  blueprint  or  a  topographic  map  are  examples 
of  cognitive  tasks.  If  a  task  involves  interpretation,  problem¬ 
solving,  or  prediction,  it  is  cognitive. 

An  interesting  element  of  cognitive  tasks  is  that  the  performer- 
perceived  difficulty  of  doing  the  task  may  not  correlate  very 
strongly  with  the  difficulty  of  learning  to  do  the  task.  For 
instance,  some  people  are  "social  geniuses."  They  always  seem 
to  know  the  right  thing  to  say,  always  make  those  around  them 
feel  comfortable  and  important,  seem  to  thrive  in  any  social 
environment,  and  so  on.  So,  you  ask  them,  "how  do  you  do 
that?"  and  they  respond,  "Do  what?"  The  sophisticated  skills 
they  display  have  been  learned  and  developed  unconsciously, 
and  the  performers'  self-  perception  is  of  effortlessness. 
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Which  raises  a  couple  of  important  issues  for  analysis  of 
cognitive  tasks:  Automaticity  and  identifiability. 

Automaticity  occurs  in  skilled  performance  when  actions  that 
once  took  thought  have  been  performed  so  frequently  that  they 
get  "compiled"  into  automatic  responses  that  need  no 
conscious  awareness.  This  is  very  good  for  the  performer,  but 
very  difficult  for  the  analyst  (namely,  you).  The  reason:  People 
don't  necessarily  know  what  they're  really  doing. 

A  case  study  will  show  the  problems.  Prospector  was  (is)  a 
prominent  mid-80's  expert  system  for  oil-field  discovery.  When  it 
was  first  being  built,  the  knowledge  engineers  focused  on 
asking  the  subject  matter  experts  (SMEs)  how  they  did  their 
job,  then  on  building  a  system  that  did  the  same  thing. 
Prospector  was  blessed  with  fabulous  SMEs;  one  of  them  had 
written  the  primary  text  in  the  field.  These  were  SMEs  who 
could  explain  what  they  did  with  exceptional  clarity,  so  the  first 
version  of  Prospector  was  completed  with  very  high  hopes.  Built 
by  a  highly  competent  team  from  the  Stanford  AI  lab,  it 
scrupulously  reproduced  what  the  SMEs  reported  that  they  did. 
It  stank. 

This  caused  serious  consternation  among  the  knowledge 
engineers,  not  surprisingly.  Where  had  they  gone  wrong?  The 
SMEs  really  were  true  experts;  when  they  did  the  task,  they  got 
results.  Why  was  the  program  so  bad? 

Confused  but  undaunted,  they  returned  to  the  SMEs.  Aware  of 
work  by  Herbert  Simon  and  Anders  Ericcson  on  protocol  analysis 
(which  we'll  discuss  below  as  a  cognitive  task  analysis 
technique),  this  time  the  knowledge  engineers  came  to  watch 
the  SMEs  work,  video  cameras  in  hand  (and  this  was  eariy-70's, 
well  before  camcorders  were  commonplace).  Tapes  in  hand, 
they  returned  to  their  lab  and  performed  a  protocol  analysis,  an 
extremely  painstaking  frame-by-frame  analysis  of  everything 
the  SMEs  were  doing.  After  a  large  sweat  investment,  they  hit 
the  jackpot.  In  case  after  case,  the  SMEs  had  said  that  to  make 
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decision  A,  they  always  looked  at  X.  But  examination  of  the 
tapes,  measuring  things  like  eye  dwell  on  various  parts  of  the 
SMEs'  data  screens,  showed  that  when  making  decision  A  they 
barely  glanced  at  X,  but  they  never  made  the  decision  without 
looking  carefully  at  the  value  for  Y.  The  SMEs  simply  weren't 
aware  that  they  were  doing  it. 

Prospector  was  revised  using  the  new  knowledge,  and  it 
worked  very,  very  well.  The  essence  of  the  story  is  that  sincere, 
committed,  articulate  SMEs,  the  very  best  in  their  field,  told  the 
knowledge  engineers  the  wrong  things.  The  SMEs  believed  that 
was  how  they  worked;  that  was  how  the  subject  was  taught, 
how  it  was  written  about  in  technical  articles,  how  everybody 
knew  it  was  done.  It  took  extraordinary  diligence  and 
cleverness  on  the  part  of  some  very  smart  knowledge 
engineers  to  discover  how  it  was  really  done. 

The  message  for  cognitive  task  analysis  is  not  a  simplistic 
"watch  what  they  do,  not  what  they  say."  After  all,  the 
explanatory  component  (what  they  say)  is  critical  for  many 
applications,  and  there  is  no  a  priori  way  to  know  the  degree 
to  which  the  SMEs  in  a  particular  field  really  can  articulate  the 
underlying  processes.  The  lesson  is  more  complex.  Cognitive 
task  analysis  is  a  process  of  building  a  model  of  competence, 
and  the  criteria  for  success  are  empirical. 

If  knowledge  is  the  core  element  of  cognitive  tasks,  why  not 
simply  categorize  task  material  with  respect  to  the  knowledge 
outcomes  (as  described  in  the  section  on  The  Nine  Step 
Approach)  and  base  decisions  on  training  approach  on  the 
results? 

The  knowledge  outcomes  are  problematic  from  two 
perspectives.  First,  it  is  not  consistently  clear  what  they  refer 
to:  The  task,  the  cognitive  process,  or  the  result  of  the  cognitive 
process.  The  name  suggests  the  result,  but  the  specific 
category  names  sometimes  suggest  the  operation  of  the 
cognitive  process  (discrimination),  sometimes  what  the  process 
operates  on  (concrete  concepts),  sometimes  the  nature  of  the 
thing  to  be  learned  or  applied  (definitional  concepts), 
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sometimes  the  formal  representational  structure  of  the 
knowledge  (rules),  sometimes  the  characteristic  things  people 
seem  to  do  in  managing  certain  classes  of  task  (cognitive 
strategies). 

But  even  if  we  take  "knowledge  outcomes"  as  fuzzy-but-usable 
concepts  of  things  people  do  when  they  perform  tasks  (apply 
discrimination,  execute  rules,  understand  definitional  concepts, 
etc.),  are  they  then  adequate  as  guidelines  for  selecting 
approaches  to  training? 

Maybe  yes,  maybe  no.  John  Anderson  raises  the  identifiability 
problem:  Membership  in  a  given  knowledge  category  can  arise 
from  multiple  sources  and  have  multiple  implications,  which 
category  membership  in  itself  does  not  specify.  Said  more 
simply,  the  same  apparent  cognitive  behavior  can  arise  from 
multiple  sources,  and  you  can  never  be  absolutely  sure  you've 
identified  the  right  one. 

Figuring  out  the  right  one  is  significant  for  training,  though.  For 
example,  many  people  we  fooled  by  Joe  Weizenbaum's  ELIZA 
program  (1969),  which  used  a  simple  keyword  pattern  matcher 
to  mimic  a  Rogerian  psychotherapist.  ELIZA  appeared  to  show 
the  same  knowledge  outcome  as  a  real  Rogerian  therapist,  but 
the  "training"  Weizenbaum  gave  ELIZA  was  drastically  different 
than  that  one  would  provide  any  real  therapist  trainee.  (By  the 
way,  Weizenbaum  himself  was  appalled  that  people  reacted  as 
they  did  to  his  program.) 

Taking  another  example,  solving  syllogisms.  The  research  of 
Johnson-Laird  and  Wason  is  persuasive  that  these  are  normally 
solved  with  a  "mental  model,"  The  mental  model  approach  to 
solving  syllogisms  isn't  very  good  (our  cognitive  capabilities 
didn't  evolve  to  solve  syllogisms),  so  people  who  want  to 
become  experts  learn  the  Venn  diagram  approach,  which  works 
much  better.  But  teaching  the  mental  model  approach  and 
teaching  the  Venn  diagram  approach  are  different:  The  first 
implies  repeated  exposure,  simple  to  complex,  with  feedback, 
and  substantial  memorization.  The  second  suggests  teaching 
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operationalization  of  a  concept. 

In  general,  the  identifiability  problem  means  that  substantial 
experimental  evidence  is  needed  to  support  a  strong  statement 
about  how  people  really  solve  cognitive  problems.  Even  for  the 
best  studied  areas,  results  are  controversial  (e.g.,  diSessa 
1993)  and  often  ambivalent  (e.g.,  Patel  et  al  1993). 

All  of  this  doesn't  mean  cognitive  tasks  can't  be  taught!  Just 
that  it  can  be  hard  to  get  it  right. 

Behavior  and  cognition  are  complimentary  approaches. 
Depending  on  the  task,  one,  the  other,  or  both  may  be 
appropriate. 
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ONE  MORE  THING:  LEVEL  OF  SKILL 

Before  (finally)  describing  how  the  various  kinds  of  task 
analyses,  there's  just  one  more  thing.  You  do  task  analysis 
because  you  want  to  train  some  task.  To  decide  how  much  and 
how  deep  a  task  analysis  to  do,  you  need  to  decide  the  target 
skill  level. 

This  may  seem  to  be  pretty  obvious,  but  it  needs  to  be  done 
with  some  specificity.  After  all,  deciding  how  much  cognitive  task 
analysis  to  do,  and  when  to  stop,  is  itself  a  cognitive  task. 

Unlike  behavioral  tasks,  where  you  can  usually  tell  when  you're 
done,  knowing  when  to  stop  a  cognitive  analysis  requires  an 
"external"  criteria. 

There  are  three  steps:  Determination  of  the  planned  trainees' 
current  skill  level,  specification  of  the  target  skill  level,  and 
planning  of  the  various  kinds  of  analyses  to  bridge  the 
difference.  Analysis  is  difficult,  time-consuming,  and  expensive. 
You  want  to  do  the  right  amount  of  analysis— The  goal  is 
training  appropriate  to  need. 
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INDIVIDUAL  TRAINING:  BEHAVIORAL  TASKS 

There  are  a  lot  of  different  flavors  of  task  analysis.  Three 
approaches  cover  a  fair  amount  of  ground,  and  taken  together 
provide  reasonable  coverage:  Procedural  analysis,  fault-tree 
analysis,  and  performance  analysis.  Each  in  turn. 
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PROCEDURAL  ANALYSIS 

Procedural  analysis  is  the  most  basic  form  of  task  analysis,  the 
analysis  of  the  observable  procedure  used  to  accomplish  a  task. 
The  precondition  for  procedural  analysis  is  that  the  task  can  be 
analyzed  sensibly  into  behavioral  steps  that  form  a  consistent 
sequence,  that  the  sequence  is  straightforward  enough  that, 
once  recorded,  it  can  be  followed,  and  that  following  the 
sequence  will  result  in  task  completion  and  success. 

There  are  five  steps  in  procedural  analysis: 

1.  Write  down  the  terminal  objective(s)  and  the  standards  of 
performance.  Terminal  objectives  are  just  whatever  is 
supposed  to  be  accomplished.  Performance  standards  can 
include  completeness,  speed,  accuracy,  efficiency,  or  other 
issues  as  applicable.  If  there  are  important  intermediate 
objectives,  do  the  same  for  each  of  them.  Make  both  the 
objectives  and  the  standards  as  specific  and  measurable  as 
possible. 

2.  Find  a  competent  task  performer,  in  a  real-world  task 
environment. 

3.  Follow  him  or  her  around  and  record  all  of  the  task  steps,  in 
order.  This  step  can  be  as  simple  as  having  the  performer 
simply  describe  the  task,  or  as  elaborate  as  videotaping 
multiple  performances  and  performing  a  protocol  analysis 
on  the  content  of  the  tapes. 

4.  Convert  the  recorded  procedure  into  a  step-by-step 
procedure  description.  Jonassen  et  al  recommend  reviewing 
these  questions  to  determine  whether  you  have  done  an 
adequate  job:  "Have  all  operation  and  decision  steps  been 
included  in  the  outline?  Have  all  branches  from  decision 
points  been  included?  Is  each  step  of  the  procedure  a 
discrete  step— that  is,  a  separate  action,  and  not  part  of 
the  next  step?  Is  the  step  size  appropriate,  and  the  same 
size  across  all  steps?  Are  the  steps  described  as 
observable  performances?" 

5.  Test  and  validate  the  procedural  description— Actually  use  it 
and  determine  what  is  left  out,  what  is  unclear,  and  so  on. 
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To  perform  Step  5,  the  two  additional  analyses— fault-tree  and 
performance— can  be  helpful. 
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FAULT-TREE  ANALYSIS 


Fault-tree  analysis  is,  as  its  name  suggests,  an  analysis  of  what 
can  go  wrong.  First,  we'll  present  "standard"  fault-tree  analysis, 
then  a  modified  version  that  takes  advantage  of  work  already 
done  in  a  procedural  analysis. 

Before  continuing,  though,  a  reminder  that  we're  still  in  the 
realm  of  behavioral  tasks— cognitive  task  analysis  is  a  different 
(complimentary)  activity. 

Standard  fault-tree  analysis  distinguishes  three  levels:  Mission, 
function,  and  task.  The  mission  is  the  purpose  of  the  system, 
what  the  system  under  study  does.  The  functions  are  how  it 
does  it.  The  tasks  are  how  the  functions  are  achieved.  For 
instance,  the  mission  of  steam  turbine  maintenance  is  to  keep 
the  turbine  producing  power.  The  functions  of  maintenance  are 
to  examine  and  test  the  major  components  of  the  turbine  at 
regularly  scheduled  intervals,  and  to  replace  components  as 
necessary.  The  tasks  are  the  steps  necessary  to  examine,  test, 
repair,  and  replace  components.  The  advantage  claimed  by 
fault-tree  advocates  is  that  failures  at  the  higher  level  can  be 
analyzed  at  the  level  immediately  below,  and  by  doing  this,  a 
very  thorough  analysis  can  be  achieved.  Fault-tree  analysis  has 
up  to  nine  steps. 

1.  Define  the  system,  that  is,  the  purpose  or  goals  of  the  task 
and  context  in  which  it  is  performed,  including  relevant 
constraints. 

2.  Analyze  the  purpose,  by  listing  the  major  functions  of  the 
system,  then  the  subfunctions,  and  the  objective  tasks  that 
accompany  each  of  the  subfunctions,  and  if  appropriate  the 
subtasks  that  comprise  the  tasks. 

3.  Identify  undesired  events  or  outcomes.  What  can  go 
wrong,  at  each  functional  or  task  step? 

4.  Once  a  satisfactory  list  is  compiled— "satisfactory"  will  be 
defined  by  your  SMEs— faults  should  be  ranked  in  three 
ways:  order  of  (typical)  occurrence,  likelihood,  and 
significance. 

5.  For  each  undesired  event  or  outcome,  the  level  (mission, 
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function,  or  task)  of  the  fault  should  be  determined  and 
recorded. 

6.  Identify  the  failure  "tree"  leading  to  the  undesired  event  or 
outcome.  Do  this  hierarchically:  If  the  failure  is  at  the 
mission  level,  determine  what  functional  breakdowns 
contribute  to  or  cause  the  failure.  If  it  is  at  the  functional 
level,  do  the  same  with  the  tasks.  It  is  important  to 
distinguish  at  this  point  the  failures  that  occur  due  to 
causes  outside  the  control  of  the  "system'-external 
factors  that  no  change  in  proficiency  of  the  operators  (the 
people  you're  ultimately  interested  in  training)  could  alter. 
These  aren't  "interesting"  for  the  fault-tree  analysis,  since 
no  change  in  performance  will  alter  their  occurrence.  For 
each  undesired  event  or  outcome  not  eliminated  by  the 
"not  under  our  control"  test,  continue  to  decompose  the 
failure  at  progressively  lower  levels.  Decomposition  stops 
on  a  particular  path  when  the  contributing  factors  or 
causes  need  no  further  decomposition— a  judgement  call, 
but  often  pretty  obvious.  Continue  this  process  until  there 
is  a  hierarchical  "tree"  extending  under  each  failure. 

7.  Specify  the  relationship  between  the  various  contributing 
factors  to  the  high-level  failures.  Is  just  one  contributing 
factor  enough  to  cause  a  breakdown?  Or,  are  there 
combinations  necessary  to  do  so?  The  fault-tree  hard  core 
like  to  describe  these  using  AND  or  OR  gates,  which  are 
logical  operators  used  to  describe  computing  hardware.  In 
real  life,  you  might  need  to  specify  the  degree  to  which  a 
breakdown  occurs,  the  threshold  at  which  a  certain  factor 
becomes  significant,  and  all  sorts  of  messy  real  world 
conditions. 

8.  Repeat  steps  6.  and  7.,  as  necessary,  until  an  adequate 
fault  tree  is  developed.  "Adequacy"  is  as  judged  by  your 
SMEs. 

9.  Analyze  the  tree.  Fault-tree  purists  label  the  nodes  in  the 
tree  (A,  B,  C...,  with  contributory  nodes  labelled  AA,  AB, 
etc.).  Then,  they  quantify  the  nodes,  marking  a  numerical 
percentage  for  each  failure  event  to  each  system-level 
failure;  mark  a  confidence  level  in  the  percentages;  quantify 
the  frequency  rate  for  the  breakdown;  and  evaluate  the 
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likelihood  of  changing  or  rectifying  each  of  the  "bottom" 
level  events.  Note  that  this  quantification  is  a  good 
example  of  "numerical  bias"-the  tendency  to  believe  in  the 
power  of  numbers,  even  though  you  know  you  just  made 
them  up.  However,  this  approach  may  still  have  utility  by 
simply  enforcing  a  thorough  traversal  of  the  fault  tree  and 
an  analysis  of  how  easy  or  possible  it  is  to  fix  common  or 
especially  important  errors.  This  last  result  then  contributes 
to  the  development  of  the  training  plan. 

Fault-tree  analysis  has  been  presented  as  though  no 
procedural  analysis  had  been  done  first.  If  you  do  procedural 
analysis  first  (recommended),  you  can  save  a  lot  of  time  by 
simply  stopping  at  each  of  the  procedural  steps  and  asking, 
what  can  go  wrong  at  this  point?  Then  repeat  this  process, 
focusing  on  the  links  between  the  steps.  The  fault  tree  analysis 
can  proceed,  then,  from  the  identified  problems  for  each  of  the 
steps  and  for  the  "steps"  between  the  steps. 

There  is  a  slight  risk  in  this  approach  of  missing  "emergent" 
faults,  that  is,  faults  that  emerge  only  when  some  combination 
or  sequence  of  events  occur.  There  are  plenty  of  real-world 
failures  that  occur  not  because  of  some  particular  fault  but 
simply  because  of  the  accumulation  of  small  variations.  In  some 
cases,  not  one  of  the  variations  might  actually  be  called  a  fault, 
but  the  combination  of  a  sequence  of  such  variations  is  enough 
to  throw  the  whole  system  or  process  off.  However,  if  special 
attention  is  paid  to  the  fault  analysis  of  the  terminal 
objective(s)  of  the  task,  this  should  be  a  very  slight  risk. 
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PERFORMANCE  ANALYSIS 

The  third,  complimentary,  task  analysis  approach  is  performance 
analysis.  This  is  akin  to  fault  analysis,  but  focuses  explicitly  on 
the  human  performance  that  underlies  task  performance.  In 
contrast  to  fault  analysis,  which  may  identify  deficiencies  in 
system  design  or  resource  supply  as  easily  as  human  error, 
performance  analysis  focuses  explicitly  on  how  the  people  do 
the  job.  In  addition,  it  is  distinguished  from  fault-tree  analysis 
by  its  interest  not  only  in  actual  errors  but  in  levels  and 
standards  of  performance.  Since  in  many  training  situations,  the 
interest  lies  in  enabling  the  performers  to  do  their  best,  not  just 
to  meet  a  minimum  performance  "floor",  interest  in  levels  of 
performance  is  useful.  The  steps  for  performance  analysis  are 
straightforward. 

1.  Decide  whether  there  is  a  performance  problem.  This  can 
either  be  with  respect  to  actual  errors,  or  with  respect  to 
quality  of  performance.  For  example,  a  toxic  containment 
crew  may  be  required  to  arrive  at  the  scene  of  a  spill  with 
fifteen  minutes.  However,  the  crew  boss  may  decide  that 
for  a  particular  area,  the  crew  should  be  able  to  arrive 
within  ten  minutes.  Arriving  in  fifteen  minutes,  then,  is  not 
an  error,  but  it  may  be  a  performance  issue  the  crew  boss 
wants  to  address. 

2.  Determine  how  you  know  the  problem  exists.  Especially 
where  there  is  a  chance  that  there  will  be  a  dispute  about 
the  appropriate  standards  of  performance,  documenting 
the  evidence  that  indicates  a  performance  problem  is 
critical, 

3.  Determine  which  performance  element  is  causing  the 
problem.  What  is  happening,  or  not  happening,  to  cause 
the  performance  issue?  At  this  point,  it  is  important  to 
distinguish  the  possible  from  the  impossible,  and  to  be 
certain  that  the  identifications  are  reasonable  with  respect 
to  the  levels  of  skill  and  talent  expected  of  the  performance 
population. 

4.  Determine  where  the  locus  of  the  problem  lies.  In  the 
performance  of  a  particular  person  or  subgroup?  In  the 
interactions  between  people  or  subgroups?  Does  the 
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problem  only  arise  in  particular  situations?  Is  the  problem 
intermittent?  If  so,  is  that  because  the  standard  is  at  the 
edge  of  the  performers'  competence,  or  because  the 
performers'  vary  significantly  in  performance?  And  so  on. 

5.  Describe  the  standard  of  performance  for  the  problematic 
task.  If  you  have  done  procedural  analysis,  you  will  already 
have  this. 

6.  In  cases  where  there  are  multiple  outputs  from  a  particular 
performance  step,  specify  which  output  is  not  adequate. 

7.  Figure  out  why  there  is  a  problem.  Jonassen  et  al  suggest 
three  categories  of  things  to  consider  for  probable  causes: 

•  Capability  causes,  like  skills,  knowledge,  abilities,  or 
physical,  emotional,  or  mental  capacities 

•  Motivational  causes,  like  inadequate  incentives  or  rewards, 
conflicting  goals  or  expectations,  etc. 

•  Environmental  problems,  such  as  inadequate  resources  or 
supplies. 

8.  For  probable  causes  in  the  first  two  cases,  consider 
possible  solutions.  These  will  likely  divide  into  causes  that 
can  be  ameliorated  by  training,  and  causes  that  can't.  For 
instance,  if  there  are  motivational  problems  caused  by  (for 
instance)  unfair  rewards,  or  rewards  that  are  not 
correlated  with  task  performance  (e.g.,  advancement  purely 
on  seniority),  no  amount  of  training  will  change  this. 
Depending  on  the  context,  you  may  or  may  not  be  in  a 
position  to  recommend  change  in  this  kind  of  case. 

However  that  may  be,  it  is  important  to  take  the  realistic 
training  opportunities  away  from  this  analysis.  The  causes  of 
subpar  performance  that  can  be  changed  by  training  should  be 
integrated  into  the  picture  that  has  been  developed  with  the 
previous  two  task  analysis  approaches. 


i  • 
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BEHAVIORAL  TASK  ANALYSIS:  WRAP-UP 


There  are  many  other  task  analysis  approaches:  Learning 
hierarchy  analysis,  component  display  theory,  mathetics, 
behavioral  analysis,  and  so  on.  The  three  above  are  suggested 
as  representing  a  reasonable  "spanning"  set,  that  will  give  (for 
tasks  that  can  be  described  adequately  by  their  observable 
steps)  a  good  description  of  what  the  task  is,  how  it  is  done, 
how  it  can  go  wrong,  and  what  performance  elements  must  be 
present,  at  what  level  of  competence,  for  it  to  go  right.  In 
addition,  these  three,  taken  together,  will  also  do  the  useful  job 
of  marking  what  performance  elements  are  not  amenable  to 
training. 
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COGNITIVE  TASKS 


Cognitive  task  analyses  are  powerful,  but  the  various 
techniques  can  also  be  expensive,  and  if  the  training  target  is 
at  a  high  level  of  expertise  they  can  require  significant 
development  of  domain  expertise  on  part  of  the  analyst.  To 
select  the  best  cost/benefit  area  within  a  task,  it  is  common  to 
look  at  the  intersection  between  the  importance  of  different 
task  areas  and  task  areas  that  show  significant  differences  in 
skill  between  high  and  low  performers. 

Most  forms  of  cognitive  task  analysis  have  the  SME  think  out 
loud  as  he  or  she  performs  the  task  (or  the  subtasks  that  have 
been  identified  as  having  the  highest  payoff).  Protocol  analysis 
is  uses  this  approach,  along  with  many  other  means  of  tracking 
what  the  SME  does  (Ericsson  &  Simon  1984).  Analysis 
techniques  that  focus  on  elements,  attributes,  and  relationships 
in  the  problem  domain,  and  those  that  use  various  types  of 
cases  -  the  typical  case  technique,  the  tough  case,  interesting 
case,  prospective  case,  and  retrospective  case  techniques,  and 
the  critical  incident  technique  -  try  to  make  covert  thought 
explicit  and  overt. 

We  will  describe  five  cognitive  task  analysis  techniques: 
Cognitive  goals  analysis,  information  needs  analysis,  repertory 
grid  analysis,  laddering,  and  semiotic  analysis.  Before  that, 
though,  we  will  reveal  the  Fundamental  Law  of  Cognitive  Task 
Analysis. 
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THE  FUNDAMENTAL  LAW  OF  COGNITIVE  TASK 
ANALYSIS 

It  takes  longer. 

Expressed  more  formally,  for  any  given  schedule  S,  Time 
(cognitive  task  analysis)  =  S  +  delta  n,  where  n  =  "more." 
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GOALS  ANALYSIS 


Goals  analysis  begins  with  the  interviewer  asking  the  SME  to 
define  the  top-level  goal  for  a  particular,  representative 
scenario.  The  goal  is  then  broken  down  into  subgoals  and 
actions  that  can  be  executed  to  move  from  one  goal  state  to 
another.  The  elicitation  of  subgoals  and  actions  helps  the 
interviewer  build  both  a  declarative  network  representation  of 
the  problem  elements  and  a  set  of  production  rules 
representing  the  goal/subgoal  relationship.  The  session 
continues  until  the  scenario  is  completely  "unpacked"  in  terms 
of  its  goals  and  actions. 

Multiple  goal  analyses  sessions  are  held,  each  with  a  different 
scenario,  until  a  good  spanning  set  of  representative  scenarios 
have  been  covered,  that  is,  until  enough  of  the  task  has  been 
covered  that  the  training  need  will  be  able  to  be  addressed. 

In  the  best  of  all  possible  worlds,  multiple  SMEs  will  participate 
in  independent  goal  analysis  sessions.  This  minimizes  the  risk  of 
a  false  reading  from  encountering  a  SME  with  an  idiosyncratic 
approach  to  the  task. 

At  that  point,  the  training  designer  does  three  things: 

1.  Distinguish  cognitive  from  behavioral  goals.  This  can  be  a 
highly  productive  activity  for  identification  of  gaps  in  the 
data  gathering.  By  cross-correlating  the  behavioral  and 
cognitive  goals,  gaps  in  the  latter  can  be  identified.  Goal 
analysis  assumes  purposeful  behavior.  Therefore,  every 
behavioral  step  should  be  motivated  by  a  corresponding 
cognitive  goal,  and  if  breaks  in  correspondence  are  found, 
the  cognitive  goal  elicitation  can  be  rejoined  to  repair  the 
gap. 

2.  Analyze  cognitive  goal  tree  overlap  (common  subgoal 
trees).This  highlights  the  recurring  cognitive  structures 
within  the  domain. 

3.  Analyze  the  underlying  reasoning  strategies.  For  example, 
how  tenaciously  are  particular  kinds  of  subgoals  pursued? 

Is  there  anything  other  than  goal  satisfaction  that  causes  a 
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switch  from  one  subgoal  tree  to  another?  And  how  does 
the  expert  tell  when  he  or  she  has  reached  that  point? 

How  are  assessment,  diagnosis,  planning,  and  action 
mixed?  How  does  one  know  when  a  goal  has  been 
satisfied?  What  kinds  of  subgoals  tend  to  be  pursued  in 
parallel?  How  are  cognitive  goals  distributed  across 
behavioral  steps? 

The  time  cost  of  goal  analysis  rises  with  the  complexity  of  a 
task;  the  more  complex,  the  more  time  required  to  decompose 
it  into  its  constituent  goal/subgoal  structure.  Goals  analysis 
typically  generates  a  rich  dataset  which  must  be  assessed  and, 
in  some  cases,  reduced.  Goal  analysis  assumes  that  an  expert's 
knowledge  can  be  characterized  as  a  propositional  network  of 
factual  information  operated  on  by  a  set  of  procedures.  If  the 
task  domain  is  such  that  this  assumption  is  not  viable,  then 
goals  analysis  should  not  be  used  to  elicit  knowledge. 
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INFORMATION  NEEDS  ANALYSIS 

Information  needs  analysis  approaches  a  cognitive  task  by 
cataloging  the  information  used  during  task  performance.  In 
outline,  information  needs  analysis  is  straightforward.  Simply 
track  each  step  in  the  process  and  determine  what  information 
flows  into  and  out  of  the  step.  Then  analyze  the  information  to 
inductively  build  a  model  of  the  cognitive  task. 

As  will  come  as  no  surprise,  in  real  life  it  is  not  often  this  easy. 
The  chief  issue  is  that  since  the  problem  is  to  infer  the  structure 
of  the  cognitive  task,  you  don't  often  get  to  start  with  a  clear 
notion  of  the  sequence  of  cognitive  steps  -  you'd  be  starting 
with  a  good  chunk  of  the  thing  you're  trying  to  infer. 

For  this  reason,  information  needs  analysis  is  a  nice  compliment 
to  goals  analysis.  The  goal-tree  can  be  used  to  structure  the 
information  needs  analysis,  with  queries  centering  on  each  of 
the  goals  and  subgoals. 

Some  points  to  incorporate  in  an  information  needs  analysis: 

•  The  obvious  thing  to  ask  the  SME  is  what  information  is 
necessary  coming  into  a  subgoal,  and  what  information  is 
produced  during  the  process  of  satisfying  the  subgoal.  Both 
the  information  content  and  its  form  need  to  be  recorded. 
Recalling  the  discussion  of  distributed  representation,  the 
form  of  the  representation  is  significant. 

•  The  less-obvious  thing  to  ask  is  why  the  information  is 
necessary  or  useful.  This  can  help  uncover  further 
information,  and  it  can  also  reveal  subgoals  that  had  not 
yet  been  articulated. 

•  This  process  is  recursive  -  it  is  important  to  understand 
what  information  (and  knowledge)  the  SME-stated 
information  depends  on  for  its  utility.  Usually  information  is 
useful  to  the  extent  that  it  allows  us  to  make  distinctions 
and  to  discriminate  between  possible  choices  or  courses  of 
action.  But  such  distinctions  usually  depend  upon  prior 
supporting  information  and  knowledge,  sometimes  a  great 
deal  of  it.  The  value  of  pursuing  this  is  that  the  elements  of 


Syukhtun  Research,  Inc. 


112 


this  information  and  knowledge  that  are  task-specific  are 
also  likely  to  be  those  that  need  to  be  trained. 

•  You  can't  necessarily  trust  your  SME.  Not  through  evil 
intent;  SMEs  often  go  out  of  their  way  to  be  helpful.  The 
problem  is  a  form  of  "repetition  blindness."  If  an  SME 
performs  a  task  for  long  enough,  some  information  issues 
will  become  so  "natural"  in  the  SME's  eyes  that  they  will 
fade  from  conscious  view.  The  SME  may  be  using 
information  from  the  environment  without  being  conscious 
of  it.  One  advantage  of  recursively  following  information 
patterns  is  that  these  unconscious  perceptions  can  be 
discovered  and  articulated. 

•  Don't  forget  the  triggers.  In  many  cognitive  tasks,  the  first 
(and  sometimes  most  important)  step  is  to  notice  or 
determine  that  there  is  a  need  to  perform  the  task.  It  is 
easy  in  information  needs  analysis  to  neglect  elicitation  of 
the  information  required  to  initiate  the  task. 
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REPERTORY  GRIDS  &  LADDERING 


The  repertory  grid  technique  (Kelly  1955;  Boose  1984;  summary 
in  Boose  1990)  uses  a  two-step  approach  to  eliciting  the 
underlying  organization  of  knowledge  of  a  domain. 

First,  SMEs  are  presented  with  triads  of  elements  from  the 
domain,  and  asked  to  group  two  of  them  in  contrast  to  the 
third.  These  similarity  judgements  group  and  differentiate 
elements  of  the  domain.  Various  combinations  are  presented  to 
the  SME  until  he  or  she  believes  that  a  spanning  set  has  been 
considered.  Because  similarities  often  result  from  similar  values 
on  an  attribute  and  thus  can  be  used  to  identify  the  attributes, 
the  product  of  this  activity  is  a  set  of  dimensions  that  subjects 
use  to  organize  and  understand  the  stimulus  set.  The  second 
part  of  the  first  step  is  the  naming  of  the  dimensions  and  the 
rating  of  the  domain  elements  with  respect  to  the  dimensions 
(thus  the  grid  -  dimensions  by  cases).  That  is,  once  the  SME  has 
established  that  A  and  B  are  more  similar  to  one  another  than 
they  are  to  C  along  dimension  X,  then  elements  D,  E,  and  F  are 
also  ranked  on  dimension  X.  The  ratings  are  submitted  to 
correlational  analysis  to  determine  the  relationship  between 
dimensions. 

The  second  half  of  the  repertory  grid  method  uses  simple 
ratings  to  assign  values  to  the  attributes  (dimensions)  obtained 
in  the  first  half  of  the  method.  These  ratings  illuminate  the 
structure  of  the  dimension,  such  as  binary  (yes/no),  continuous, 
discrete,  directional,  and  so  on. 

Laddering  is  a  third,  somewhat  independent  step  that 
augments  repertory  grids.  Using  selected  individual  elements 
and  hierarchical  discrimination  questions,  different  abstraction 
levels  within  the  SME's  mental  model  can  be  identified.  In 
English,  this  means  you  ask  the  SME  questions  like,  "A  is  an 
example  of  what?"  or  "Which  of  these  elements  is  an  example 
of  A?"  By  pursuing  this  kind  of  inquiry,  you  can  get  a  good 
notion  of  the  different  levels  of  the  domain. 
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If  the  cognitive  task  involves  a  significant  amount  of 
classification  (most  diagnostic  tasks  do),  repertory  grids  and 
laddering  are  effective  and  (comparatively)  Inexpensive 
techniques. 


Syukhtun  Research,  Inc. 


SEMIOTIC  ANALYSIS 


Semiotics  have  a  funny  reputation  in  science,  probably  because 
the  methods  originated  in  the  study  of  literature.  Just  because 
some  French  intellectuals  invented  them  doesn't  mean  the 
techniques  aren't  useful,  though.  To  illustrate  semiotic  analysis, 
we'll  use  a  framework  proposed  by  Lemieux  and  Bordage 
(1992).  Similarly  to  the  declarative/procedural  distinction  in 
psychology,  semioticists  distinguish  knowledge  and  the 
operations  that  can  be  performed  upon  it. 

Lemieux  and  Bordage  propose  three  levels  of  knowledge: 

1.  Constituent  units  are  the  various  pieces  of  the  domain.  For 
example,  in  medicine,  symptoms,  tests,  conditions,  and 
states  would  be  constituent  units. 

2.  Morphological  units  form  the  grammar  of  the  domain,  that 
is,  how  things  fit  together  and  what  they  mean. 

3.  Mental  operations  are  the  productive  combinations  of 
constituent  and  morphological  units  to  achieve  domain 
goals. 

Lemieux  and  Bordage  name  five  categories  of  mental 
operations  that  together  comprise  competence: 

1.  Definitional  operations  are  concerned  with  the  form  and 
substance  of  the  domain  content. 

2.  Classificatory  operations  locate  the  content  within  a 
taxonomy  (used  in  the  general  sense  of  a  system  of 
organization). 

3.  Hierarchical  operations  are  concerned  not  with  level  in  a 
taxonomy,  but  with  relative  importance.  The  hierarchy  is  a 
salience  hierarchy,  which  is  assumed  to  be  content- 
sensitive  and  dynamic. 

4.  Associative  operations  are  the  process  of  putting  material 
in  context,  e.g.,  similarity,  analogy,  difference,  equivalence, 
compatibility,  and  so  on. 

5.  Generative  operations  are  about  causal  relations  and 
inference. 

A  semiotic  analysis  proceeds  in  five  stages: 
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1.  Collect  protocols.  This  is  the  basic  data  gathering.  It  can 
proceed  by  presentation  of  various  cases  in  an  interview 
setting  (as  mentioned  above)  or  by  field  observation. 

2.  Code  the  transcripts  according  to  knowledge  level. 

3.  Analyze  the  coded  transcripts  by  mental  operation. 

4.  Extract  the  patterns  Of  sequence  and  dependency. 

5.  Complete  the  characterization  of  the  domain. 

Five  easy-to-name  steps.  Unfortunately,  like  other  approaches 
involving  close  analyses  of  protocols,  it  can  be  extremely  labor- 
intensive.  The  benefit  is  that  a  very  thorough  analysis  can  be 
performed,  and  the  product  of  the  analysis  is  in  a  form  that  is 
easier  than  most  to  translate  into  training  goals. 
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TEAM  TRAINING 

What  emergent,  trainable  characteristics  distinguish  teams  from 
the  actions  of  their  members?  Team  training  often  focuses 
predominantly  on  development  of  individual  skills  in  a  team 
setting,  in  effect  having  the  constituent  individuals  each  practice 
their  contributory  skills.  Even  something  as  intrinsically  team- 
oriented  as  compensatory  behavior  can  be  seen  as  an 
individual  skill.  But  what  about  team  skills  which  are  not 
necessarily  individual  skills  exercised  in  a  team  setting?  Do  such 
skills  exist?  How  could  they  be  trained? 

It  is  known  that  for  many  NP-hard  problems  (where  the  number 
of  possible  solutions  is  exponential  and  the  minimal  solution 
path  requires  traversing  all  possible  points  in  the  solution 
space),  excellent  but  non-optimal  solutions  can  be  found  by 
techniques  that  trade  local  optimality  for  global  goodness.  To 
see  why  this  works,  consider  design  of  a  complex  artifact,  a  jet 
airliner  for  example.  The  optimal  solution  for  the  design  of  the 
hydraulics  and  the  optimal  design  for  the  passenger 
compartment  can  easily  conflict.  Even  more  basically,  the  need 
for  airframe  strength  is  likely  to  conflict  with  the  need  for 
airframe  lightness.  Because  problems  like  these  typically  involve 
mutual  exclusion  of  possible  substates,  optimal  solutions  are 
simply  too  difficult.  Compromises  are  necessary  to  find  good 
solutions.  As  an  old  engineering  heuristic  has  it,  "the  best  is  the 
enemy  of  the  good." 

The  implication  for  team  training  is  that  a  key  element  in 
successful  teams  is  individuals'  sublimation  of  their  own  best 
performance  in  the  pursuit  of  the  team's  best  performance.  A 
team's  best  performance  may  result  when  the  individuals  who 
make  it  up  do  not  each  provide  their  individual  best 
performance. 

This  is  not  obvious  to  everyone.  It  distinguishes  higher¬ 
performing  teams  from  lesser  ones,  and  it  is  certainly  trainable. 
It  appears  to  be  an  emergent  property,  a  team,  not  an 
individual,  skill:  It  must  be  coevolved  among  the  individual  team 
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members.  The  notion  of  coevolution  of  skill,  then,  is  a  candidate 
marker  for  emergent  team  skills  that  are  not  summary 
individual  skills.  There  is  a  lot  of  work  in  both  ecology  and  in 
complexity  theory  that  supports  this  idea.  But  it  is  speculative 
at  this  point,  based  on  field  observation  but  not  yet  on 
controlled  experimentation. 

In  the  following,  well  cover  task  analysis  for  teams,  both 
behavioral  and  cognitive.  Some  of  the  material  covered  in  the 
sections  on  individual  tasks  can  also  be  applied  to  teams  -  the 
fault  tree  analysis  in  particular,  with  its  emphasis  on  the  system 
level,  can  easily  be  applied  to  teams.  Team  behavioral  task 
analysis  first. 
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TEAM  BEHAVIORAL  TASKS 

The  first  step  in  team  behavioral  task  analysis  is  to  perform  a 
procedural  analysis  with  the  unit  of  analysis  being  the  team 
instead  of  the  individual.  As  with  individuals,  the  three 
preconditions  for  team  procedural  analysis  are  that  the  task 
comprises  behavioral  steps  that  form  a  consistent  sequence; 
the  sequence,  once  recorded,  can  be  followed;  and  following 
the  sequence  always  results  in  task  completion.  ("Always" 
being  as  absolute  or  as  probabilistic  as  is  appropriate  for  the 
task  domain.) 

The  five  procedural  analysis  steps: 

1.  Record  the  team's  terminal  and  important  intermediate 
objective(s),  and  the  standards  of  performance  for  each. 
Care  should  be  taken  that  the  objectives  of  the  team 
leader  or  key  team  members  are  not  mistaken  for  the 
overall  team  goals.  It  is  also  useful  to  record  the  functional 
role  of  each  intermediate  or  enabling  objective.  This  can  be 
used  to  distinguish  steps  whose  temporal  order  reflects  an 
underlying  dependency  relationship  and  those  whose 
ordering  is  for  convenience  or  out  of  habit. 

2.  Find  a  competent  team  that  can  be  observed  performing 
the  task  in  a  real-world  environment.  This  is  much  more 
difficult  for  teams  than  it  is  for  individuals,  especially  for 
tasks  that  are  performed  in  a  highly  dynamic  environment. 
Military,  police,  fire,  and  paramedic  teams  may  need  to 
respond  to  a  wide  range  of  circumstances,  making 
observation  of  specific  tasks  difficult. 

3.  Observe  the  team  performing  the  task  and  record  the  task 
steps.  Depending  on  the  variability  of  the  task  and  the 
dynamism  of  the  environment,  this  step  may  need  to  be 
repeated  a  number  of  times.  In  a  highly  variable 
environment,  observation  of  multiple  teams  may  also  be 
useful. 

4.  Convert  the  recorded  procedure  into  a  step-by-step 
procedure  description.  It  is  again  incumbent  on  the  analyst 
to  maintain  a  clear  distinction  between  the  team  and  the 
individuals  who  make  it  up. 
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5.  Test  and  validate  the  procedural  description.  Once  again, 
this  is  harder  for  teams  than  it  is  for  individuals,  and  it  is 
harder  for  tasks  performed  in  dynamic  environments. 

The  result  of  these  steps  will  be  a  procedural  description  of  the 
team's  task.  It  will  include  information  on  task  steps  that  can  be 
performed  in  parallel  and  steps  that  cannot. 

The  second  analytic  step  is  to  integrate  the  team  model  with 
the  various  individual  task  performance  models.  This  is  a 
somewhat  painstaking  process  of  knitting  together  and  teasing 
apart. 

1.  Match  and  connect  the  team  procedure  with  the  individual 
component  procedures.  This  is  the  knitting  together.  Note 
that  individual  tasks  will  often  cross  over  the  boundaries 
between  team  task  steps. 

2.  Analyze  the  integration  with  respect  to  the  eight  elements 
of  teamwork.  For  example,  every  team  step  has  some 
initiation  conditions,  some  reasons  why  it  is  time  to  begin 
performing  it.  To  the  extent  that  these  are  contingent  upon 
the  performance  of  multiple  individual  tasks,  then  the 
teamwork  elements  necessary  to  coordinate  those 
individual  tasks  and  signal  to  the  team  that  the  team  task 
step  can  be  begun  should  be  analyzed.  That  is  analysis  of 
team  step  initiation;  the  same  kind  of  analysis  should  also 
be  applied  to  within-step  individual-to-team  coordination 
activities.  The  more  deeply  the  task  penetrates  into  the 
realm  of  real  teamwork  (as  contrasted  with  side-by-side 
taskwork),  the  more  complex  and  difficult  this  analysis  will 
be,  but  the  more  necessary  as  well. 

This  two-stage  procedural  analysis  can  be  augmented  by  a 
performance  analysis  that  focuses  on  the  eight  team  interaction 
behaviors.  The  subject  of  this  analysis  would  be  the 
interactions  between  team  members.  This  analysis  must  do  two 
things.  It  must  examine  particular  teams  and  the  interactions 
within  them,  and  it  must  then  generalize  this  for  the  subject 
category  of  teams.  That  is,  to  be  useful  for  training,  the 
idiosyncrasies  of  a  particular  team  need  to  be  discarded,  but 
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then  general  problems  that  any  team  will  encounter  need  to  be 
preserved  and  highlighted. 

These  analyses  will  effectively  span  the  team  behavior  space. 
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TEAM  COGNITIVE  TASKS 

Cognitive  task  analysis  for  groups  is  a  new  field.  Unlike  task 
analysis,  it  hasn't  had  fifty  years  of  development,  and  its 
methods  are  more  controversial  than  those  of  its  behavioral 
counterpart. 

Not  that  behavioral  task  analysis  has  a  lot  of  solid  comparative 
studies.  Reading  the  task  analysis  literature,  one  encounters  a 
lot  of  material  that  is  presented  on  the  basis  of  "we  thought 
this  was  a  good  idea  and  we  tried  it  and  it  seemed  to  work." 
Note  that  the  authors  don't  actually  say  that.  They  usually  say 
they  found  some  failure  or  fault  in  the  other  methods  and 
invented  a  new,  improved  approach.  There  are  many  improved 
approaches;  it  is  almost  (but  not  quite)  as  difficult  to  find  an 
approach  that  isn't  "new  and  improved"  as  it  to  find  a 
controlled,  comparative  study. 

With  that  preamble,  plus  the  probably  unnecessary  reminder 
that  cognition  is  harder  to  study  than  behavior,  we  present  four 
team  cognitive  task  analysis  approaches:  Team  cognitive  goals 
analysis,  team  information  needs  analysis,  distributed 
representation  analysis,  and  team  metacognition  analysis. 

We  hope  that  over  the  coming  years,  the  needs  of  and  for  team 
cognitive  task  analysis  will  see  a  meeting  and  blending  of 
methods  from  psychology,  sociology,  and  anthropology  in  order 
that  we  might  have  much  better  methods  than  can  be 
presented  below. 
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TEAM  COGNITIVE  GOALS  ANALYSIS 

Teams  have  goals  that  individuals  share  as  members  of  a  team 
but  cannot  hold  as  individuals.  This  is  often  obscured  because 
members  of  good  teams,  and  especially  team  leaders,  so 
strongly  identify  with  the  overall  team  goal(s).  (Is  this  the 
source  of  the  odd  grammatical  convention  of  kings  and 
corporate  bosses  referring  to  themselves  in  the  plural?) 
Nevertheless  it  is  true.  No  individual  puts  out  a  canyon  fire  or 
wins  a  soccer  game. 

The  analytic  implication  is  that  team  cognitive  goals  analysis 
proceeds  on  the  same  lines  as  individual  cognitive  goals 
analysis.  The  team  as  a  whole  is  treated  (in  a  limited  sense)  as 
though  it  were  an  individual.  Top-level  goals  are  defined  for  the 
scenario,  then  unpacked  into  subgoals,  with  the  unpacking 
process  continuing  until  the  team  goals  bottom  out  by 
intersecting  the  individual  goals.  Goal  analyses  sessions  should 
be  held  with  different  scenarios  and  with  SMEs  representing 
each  of  the  roles  or  positions  used  in  the  team. 

The  cognitive  goals  should  be  clearly  distinguished  from  the 
behavioral  goals.  In  some  cases  this  will  be  obvious  on  the 
surface,  in  others  it  will  need  to  be  teased  out  -  why  is  a 
particular  team  behavioral  goal  adopted  at  a  particular  point  in 
the  task  process?  How  tenaciously  are  particular  kinds  of 
behavioral  subgoals  pursued,  and  what  cognitive  goals 
motivate  what  levels  of  behavioral  activity?  How  are  team 
cognitive  goals  distributed  across  team  behavioral  steps?  And 
so  on. 
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•  Self-reports  are  frequently  inadequate.  Task  observation  is 
necessary.  Effective  teams  may  practice  sophisticated 
information  sharing  without  being  consciously  aware  of 
doing  so.  In  particular,  information  "publication"  is  of 
interest.  By  this  we  mean  the  performance  of  some  portion 
of  the  task  in  a  way  that  makes  performance  or  markers  of 
progress  more  visible  than  is  strictly  required  by  the 
structure  of  the  task. 
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DISTRIBUTED  REPRESENTATION  ANALYSIS 

Distributed  representation  analysis  is  applicable  to  team 
cognitive  task  analysis  because  for  some  team  activities  the 
whole  of  a  thought  or  plan  is  distributed  over  the  whole 
system,  with  the  various  participants  each  only  having/being 
part  of  it,  and  perhaps  none  compassing  it  in  its  entirety. 

There  are  two  steps  in  distributed  representation  analysis.  The 
first  is  simply  to  determine  what  the  shared  representations 
are.  Some  of  this  will  be  totally  straightforward.  If  the  doctor 
writes  an  observation  on  a  chart  and  another  doctor 
subsequently  reads  it,  this  is  pretty  easy  to  observe.  Less  easy 
to  detect  are  the  unconscious  or  "incidental"  representations. 
Hutchins  gives  the  example  of  the  characteristic  placement  of  a 
pencil  stub  at  a  particular  angle  in  between  two  gauges  in  the 
cockpit  of  a  747.  The  placement  of  the  stub  records  a  reading 
taken  at  a  particular  time  and  additionally  signals  that  the 
overall  joint  process  is  proceeding  on  schedule.  As  an 
observational  task,  detecting  a  pencil  stub  placement  as  a 
significant  distributed  representation  action  is  difficult,  requiring 
as  a  minimum  both  an  alert  eye  and  a  large  amount  of  detailed 
domain  knowledge. 

The  second,  and  despite  the  example,  generally  more  difficult 
part  of  distributed  representation  analysis  is  figuring  out  what 
each  instance  means  for  the  team  cognitive  task  with  respect  to 
trainable  skills  and  capabilities.  This  is  the  painstaking  process 
of  working  out  the  function(s),  required  supporting  assumptions 
and  knowledge,  and  implications  for  each  of  the  instances. 

To  the  extent  that  the  analysis  is  with  respect  to  function,  this 
analysis  is  closely  tied  to  the  team  cognitive  goals  analysis. 
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METACOGNITIVE  GOALS  ANALYSIS 


Metacognitive  skills  monitor  and  control  cognitive  activities  in 
such  a  way  as  to  maximize  their  effectiveness.  A  wide  range  of 
mental  operations  fall  into  this  category:  Managing  focus  and 
duration  of  attention,  detecting  problems,  monitoring  progress 
and  assessing  when  a  change  in  approach  is  appropriate,  and 
so  on.  Metacognitive  skills  can  be  thought  of  as  self¬ 
management  with  respect  to  problem-solving. 

Salas  et  al  (from  whom  the  seven  of  the  eight  teamwork 
characteristics  were  taken)  discuss  three  broad  aspects  of 
teamwork.  Two,  taskwork  and  teamwork,  have  been  covered  in 
detail.  The  third  they  call  "jelling,"  by  which  they  mean  the 
degree  to  which  a  particular  team  manages  to  fit  together  and 
make  itself  more  than  the  sum  of  its  parts.  A  significant  part  of 
the  jelling  process  has  to  do  with  effective  team-level 
metacognition. 

Sternberg  identifies  four  metacognitive  skills: 

1.  Identifying  and  defining  the  problem,  including  recognizing 
that  there  is  a  problem  to  solve. 

2.  Representing  the  problem,  including  how  to  figure  out 
exactly  what  the  problem  is  and  how  it  should  be  framed. 

3.  Planning  how  to  proceed. 

4.  Evaluating  performance  as  it  occurs  and  continually 
performing  the  preceding  three  steps  during  performance. 

The  approach  to  team-level  metacognitive  analysis  is  to 
examine  the  team  as  a  whole  for  markers  of  metacognitive 
activity.  When  and  under  what  circumstances  do  each  of  the 
four  metacognitive  activities  occur?  In  what  way  do  they  change 
and  improve  team  performance? 

In  order  to  do  this,  comparative  study  of  long-term  and  newer 
teams  is  desirable.  In  some  fields  this  can  be  difficult,  for  any 
number  of  reasons:  There  are  no  long-term  teams,  long-term 
teams  are  always  composed  of  more  individually  experienced 
members,  or  long-term  working  relationships  exist  but  team 
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boundaries  shift  and  reform  enough  that  specific  on-the- 
team/not-on-the-team  assignments  are  difficult  to  make.  If, 
though,  such  comparative  studies  can  be  made  differences  in 
performance  between  otherwise  matched  teams  can  provide 
useful  material  for  analysis. 
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TEAM  COGNITIVE  TASK  ANALYSIS  WRAPUP 

Cognitive  team  task  analysis  is  a  new  activity.  The  approaches 
described  above  could  easily  be  criticized  as  having  excessive 
redundancy  and  often-vague  completion  criteria.  On  the  one 
hand,  such  criticisms  would  be  valid.  On  the  other,  a  reasonable 
picture  of  the  team  cognitive  task  can  be  developed  using  the 
methods  described. 

It  is  useful  to  remember  that  the  point  of  all  this  work  is  to  be 
able  to  design  and  implement  more  productive  training.  Even 
though  the  epistemological  elements  of  cognitive  modeling  must 
be  elucidated  for  long-term  progress  to  be  made,  it  is  not  the 
obligation  of  the  training  designer  to  do  so.  At  every  point  in 
the  team  cognitive  task  analysis  process,  it  is  fair  to  ask,  "what 
difference  does  this  make?"  If  a  reasonably  persuasive  answer 
cannot  be  given,  there  will  be  little  point  in  pursuing  the 
particular  line  of  investigation. 
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RELATION  TO  INSTRUCTIONAL  GOALS 


Having  made  it  this  far  (congratulations!  And  thank  you!),  you 
have  the  tools  with  which  to  define  the  features  and  processes 
of  your  target  training  task,  both  behavioral  and  cognitive,  for 
both  individuals  and  teams. 

If  you  have  performed  some  of  the  analyses,  especially  any  of 
the  cognitive  analyses,  you  are  probably  in  state  of  something 
like  astonishment  that  people  are  able  to  manage  the  incredible 
complexity  that  underlies  even  the  (apparently)  simplest  tasks. 

The  next  step  is  training  design.  There  are  four  routes  you 
might  take  from  here: 

1.  There  are  some  basic  elements  of  training  shared  by  all 
effective  training  programs.  These  are  introduced  in  <How 
To  Use  The  Guidebook>. 

2.  Our  expert-system-based  training  advisor  will  recommend 
training  techniques  according  to  how  you  answer  "a  few" 
(198)  questions.  These  questions  let  the  system  build  an 
abstract  characterization  of  the  task  you  wish  to  train  - 
does  it  take  place  in  a  dangerous  environment?  Does  it 
require  extended  vigilance?  Is  mutual  monitoring  of 
teammates  a  critical  element?  And  so  on.  The  system  will 
then  make  recommendations  based  on  our  survey  of  the 
current  state  of  the  art  and  on  our  own  research.  Go 
cTraining  Advisor>. 

3.  Look  at  our  research  on  teamwork.  See  what  we  asked  our 
subjects  and  why,  how  we  analyzed  the  resulting  data, 
and  what  we  found  out.  Go  cTeamwork  Research 
Findings>. 

4.  Go  directly  to  the  Guidebook,  where  you'll  find  descriptions 
of  myriad  training  approaches.  Go  <The  Guidebook>. 
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UNIVERSAL  ISSUES  IN  TRAINING 

There  are  some  issues  that  any  training  approach  will  have  to 
address: 

•  Deciding  who  to  train 

•  Prerequisite  skills  and  knowledge 

•  Expectations 

•  Establishing  an  environment  for  training 

•  Motivation 

•  Goals 

•  Feedback 

•  Reinforcement 

•  Transfer 

None  of  these  are  problems  with  definitive  solutions,  just  issues  to 
recognize  and  integrate  into  training. 
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DECIDING  WHO  TO  TRAIN 

Training  is  clearly  essential  but  must  be  used  selectively  to  meet  the 
needs  of  the  organization  as  well  the  individual.  Some  suggestions 
are  included  here  to  help  managers  to  decide  who  should  receive 
training. 

It  is  important  to  be  aware  of  the  reasons  behind  the  training.  Is  it 
job  training,  training  for  advancement,  or  new  practice  or  technology 
training?  If  the  purpose  of  training  is  based  on  set  aims  for  the 
organization,  this  will  make  trainee  selection  easier. 

Training  newcomers  is  often  the  easiest  need  to  identify.  Be  careful, 
though,  to  assess  individual  needs  especially  when  post  transfers 
are  involved.  Orientation  may  suffice. 

Job  training  can  be  initiated  when  a  need  is  identified  by  those 
performing  the  job.  It  is  important  in  this  case  to  ensure  the 
resulting  training  is  offered  to  those  in  the  areas  indicated  by  them. 

New  technology  or  major  changes  of  any  kind  should  always  be 
geared  to  key  areas  and  so  to  the  people  directly  involved.  It's  been 
noted  that  training  everyone  creates  resentment  in  those  who  are 
not  permitted  to  use  their  new  skills.  It  also  undermines  respect  for 
management. 

It's  worth  bearing  in  mind  that  people  often  associate  training  with 
job  advancement.  This  is  another  reason  to  clearly  explain  the 
motivation  for  training  and  whether  or  not  it  does  entail  a  job 
advancement.  Advancement  can  be  related  to  training  itself,  for 
example  in  the  delegation  of  further  training  responsibilities  to  the 
person  you  select  first. 

Be  sure  to  discuss  clearly  the  aims  and  expectations  of  training  with 
key  subordinates.  This  will  help  determine  who  needs  it.  Needs 
assesment,  making  training  seem  important,  and  choices  about 
training  are  related  topics  which  should  help  in  this  kind  of  decision 
making. 
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PREREQUISITE  SKILLS  AND  KNOWLEDGE 

To  train  effectively,  knowing  the  abilities  of  the  learners  as  they 
enter  training  is  necessary.  It  both  saves  time  and  communicates 
respect.  Inadequate  assessment  and  training  pitched  at  the  wrong 
level  can  if  too  easy  lead  to  boredom  and  if  too  difficult  to 
breakdown  of  the  learner's  sense  of  personal  control  ("I  can't  learn 
this")  and  thence  motivation  ("so  why  should  I  try?"). 

Assessment  is  the  evaluation  of  a  person's  strengths  and 
weaknesses  on  a  certain  set  of  tasks,  in  order  to  determine  what 
things  they  need  to  work  on  (either  on  their  own  or  through 
training).  What  is  presented  below  is  not  how  to  select  tasks  to  be 
assessed,  nor  is  it  how  to  assess  them,  nor  how  to  design  training 
based  on  the  needs  -  it  is  how  to  present  needs  assessment. 
Selection  of  areas  for  assessment  and  design  of  assessment 
instruments  are  domain  specific.  How  to  design  training  is  covered 
later.  The  following  is  about  how  to  present  assessment  results. 

Because  we  don't  like  to  hear  what  is  wrong  with  us,  we  tend  to 
want  to  discredit  the  source  or  in  some  other  way  make  it  so  that 
we  don't  have  to  believe  what  the  assessor  is  saying.  It  is  important 
to  present  training  or  learning  needs  in  such  a  way  that 

1.  The  learners  don't  feel  the  urge  to  do  this. 

2.  They  don't  have  a  basis  for  doing  it. 

If  you  do  this  poorly,  they  may  revolt  against  the  training  or  dismiss 
the  information. 

In  Noe's  list  of  necessary  conditions  for  learning,  the  first  entry  is 
the  belief  that  assessment  of  personal  strengths  and  weaknesses  is 
accurate.  There  are  four  things  you  can  do. 

•  Tell  the  trainees  why  the  assessment  is  being  done.  Be  honest. 

•  Involve  trainees  in  the  needs  assessment.  Have  them  help 
figure  out  what  needs  to  be  trained  and  how  best  to  train.  This 
input  must  be  used,  though  -  it  backfires  seriously  if  you  ask  for 
their  opinions  only  to  ignore  them. 

•  Tell  people  how  the  needs  assessment  was  completed.  This  will 
reduce  suspicion,  fear,  animosity  towards  the  training. 

•  Make  sure  that  the  information  is  accurate  (credible)  and  that  it 
comes  from  a  trustworthy  source.  If  the  employees  or  trainees 
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have  some  a  priori  reason  to  believe  that  you  aren't 
trustworthy  -  there  are  plenty  of  Dilbertian  reasons  this  could 
be  the  case  -  then  this  needs  special  care. 

If  employees  do  not  understand  why  and  how  their  needs  were 
diagnosed,  or  feel  that  the  information  (or  its  source)  is  not  credible, 
they  will  be  resistant  to  change  -  both  in  resisting  the  training  and  in 
not  making  use  of  the  information  provided  about  their  comparative 
strengths  and  weaknesses. 

It  is  important  to  frame  the  results  of  the  assessment  in  a 
constructive  way.  Avoid  making  it  competitive  -  finding  that  you  are 
labeled  with  a  particular  "weakness"  whereas  some  other  person 
isn't  could  lead  to  feelings  of  insecurity  or  resentment  both  towards 
them  and  the  suggestion.  On  that  note,  make  sure  that  being 
placed  in  training  does  not  become  a  signal  reduced  status  ("you 
are  a  loser"). 

But  at  the  same  time,  don't  hold  back  information  that  could  be 
useful  because  you  don't  want  to  hurt  someone's  feelings.  The 
optimal  situation  is  where  you  can  make  it  clear  to  them  why  you 
are  telling  them  -  so  they  can  get  better! 

This  brings  up  a  complex  point.  In  the  huge  majority  of  domains,  we 
assume  good  will,  a  desire  to  improve,  and  mutual  respect.  There  is 
a  traditional  exception:  Military  boot  camp.  This  is  a  special  case,  in 
which  the  intent  of  assessment  information  may  be  to  break  the 
trainee  down,  so  that  he  or  she  can  be  psychologically  remolded 
into  someone  who  will  unhesitatingly  kill  or  die  because  he  or  she  is 
told  to. 

The  more  often  needs  are  assessed,  the  less  critical  it  is  to  be 
careful  about  the  way  one  does  it  in  any  one  instance,  as  people  will 
grow  accustomed  to  it.  On  the  other  hand,  the  more  important  it 
becomes  in  general,  because  the  cumulative  effect  of  how  you  do  it 
is  greater,  and  you  may  end  up  making  people  ignore  you  all  the 
time. 

If  trainees  perceive  that  their  training  needs  have  been  accurately 
diagnosed  and  that  it  provides  useful  information  about  strengths 
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and  weaknesses,  they  will  react  favorably  to  the  assessment. 
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EXPECTATIONS 

Expectations  have  a  huge  effect  on  both  training  outcomes  and  on 
trainees'  subjective  assessment  of  training.  Giving  trainees  accurate 
expectations  of  what  is  going  to  happen  during  training,  and  what 
the  results  are  going  to  be,  is  a  very  good  idea. 

Hicks  and  Klimoski  (1987)  sent  out  either  meaningful  or  bland 
descriptions  of  the  training  that  the  people  were  about  to  engage 
In.  They  found  that  having  realistic  expectations  caused 

•  Greater  motivation  to  learn. 

•  Greater  commitment  to  attend. 

•  Greater  feeling  that  the  training  was  worthwhile. 

•  Stronger  sense  of  being  able  to  profit  from  the  training. 

Tannenbaum,  Mathieu,  Salas,  and  Cannon-Bowers  found  that 
expectation  fulfilment  was  related  to  commitment,  self-efficacy  (both 
physical  and  academic),  and  motivation  (although  the  effect  was 
smaller  for  self-efficacy).  They  also  found  that  liking  the  training  led 
to  commitment  and  physical  self-efficacy. 

Noe  found  that  trainees  who  had  their  pretraining  expectations  met 
(e.g.,  degree  of  challenge,  opportunity  to  practice)  developed 
greater  postraining  commitment,  self  efficacy  and  motivation. 

So  -  having  appropriate  expectations  and  having  them  fulfilled  is 
good.  Be  sure  that  trainees  come  into  the  training  with  accurate 
expectations,  by  telling  them  what  to  expect  (and  if  appropriate 
what  not  to  expect),  and  then,  meet  their  expectations. 

Baldwin  &  Magjuka  (1991)  found  that  if  you  enter  training  expecting 
a  follow-up  activity  or  assessment,  you  will  report  stronger 
intentions  to  transfer  training  back  to  their  jobs. 

Noe  claims  that  labeling  training  as  an  "opportunity"  produces 
better  outcomes,  by  generating  higher  learining  expectations. 
However,  care  needs  to  be  exercised.  Labeling  effects  are  real,  but 
if,  for  example,  downsizing  has  produced  a  situation  where  one 
person  is  being  trained  to  do  the  work  that  two  people  used  to  do, 
telling  him  or  her  that  it  is  an  opportunity  will  merely  rub  salt  into  the 
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wounds. 


It  is  important  to  make  sure  that  the  necessary  conditions  for  using 
the  new  skill  (equipment,  information,  environment,  etc.)  are 
present  before  beginning  training.  Immediate  application  improves 
retention,  and  conversely,  a  clear  lack  of  application  reduces 
motivation  -  why  do  the  work  of  learning  if  you  won't  be  using  the 
new  skill? 

There  may  be  situations  where  you  don't  want  trainees  to  have  full 
knowledge  of  what  is  going  to  happen  next.  It  may  be  that  they 
need  training  in  dealing  with  surprise,  or  too  much  prior  knowledge 
will  cause  heightened  anxiety  if  an  unusually  stressful  situation  is 
anticipated.  (See  the  Guidebook  sections  on  stress  training  for  more 
discussion.) 

If  trainees  know  what  to  expect  out  of  training,  they  will  feel  more 
self-assured  (in  contrast  to  feeling  unprepared  or  helpless).  As 
expectations  are  met,  they  will  feel  they  are  making  progress.  And, 
they  will  not  be  shocked,  scared,  or  anxiety-stricken  as  new  tasks, 
goals,  or  techniques  are  introduced.  If  they  know  why  they  are 
being  trained  and  what  benefit  they  can  reasonably  expect,  they  will 
be  more  willing  to  accept  the  training  as  worthwhile  and  commit  to 
doing  well. 
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ENCOURAGE  GOOD  HABITS 

Good  training  habits  are  worthwhile  instilling  in  the  trainee  to  the 
point  where  they  become  automatic.  It  helps  if  trainees  also  have 
this  as  a  conscious  aim.  Be  clear  that  you  always  expect  behaviour 
appropriate  for  training  and  use  techniques  that  encourage  this 
such  as  contracts  and  establishing  a  training  environment.  This 
helps  to  create  the  right  notion  of  what  training  is  supposed  to  be 
like. 

When  you  use  a  training  technique,  it  may  help  the  learner  if  you 
explain  what  you  are  doing.  In  this  way  he  is  encouraged  to  think 
and  act  in  a  similar  vein.  Lead  by  example,  always  taking  the 
attitude  you  want  trainees  to  take.  Ultimately,  a  trainee's  attitude 
and  actions  should  be  maintained  without  any  external  example. 

If  there  is  more  than  one  trainee,  a  new  trainee  will  take  cues  from 
the  others.This  is  also  true  vice  versa.  If  you  have  already 
established  an  environment  where  the  learners  concentrate  and 
work  hard  with  a  positive  attitude,  then  a  new  trainee  will  usually  fit 
right  in.  However,  a  new  trainee  may  sometimes  change  the 
environment  negatively,  instead  of  adapting  to  it.  In  this  case,  basic 
training  techniques,  including  socialization,  are  called  for  once  again. 

Reward  good  training  accomplishments,  but  distinguish  between 
improvement  and  performance.  Trainees  should  be  aware  that  their 
efforts  can  be  rewarded  while  their  performance  standard  is  not  yet 
satisfactory.  Rewarding  improvement  fosters  concentration  as  well 
as  good  attitudes.  Shaping  is  a  particularly  worthwhile  technique  for 
this  purpose.  The  emphasis  is  on  positive  reinforcement  as  a  more 
productive  approach  than  punishment  for  lack  of  concentration. 

As  self  mastery  increases  the  need  for  reward  declines.  Be  careful 
not  to  withdraw  rewards  too  early.  Monitor  mood  as  well  as 
achievements  to  avoid  this,  because  rewarding  progress  can  also 
aid  in  making  training  more  fun  for  the  trainee. 

Concentration  can  be  instilled  as  a  habit.  Make  sure  that  when 
practicing,  the  student  is  concentrating.  Do  not  practice  when  they 
are  not  concentrating  -  if  they  stop,  you  stop  and  make  them  start 
again.  This  may  be  difficult,  but  the  aim  is  to  always  build  in  periods 


Syukhtun  Research,  Inc. 


139 


of  consistently  high  concentration  during  training.  Formal  breaks  or 
short  times  of  less  intense  activity  should  be  clearly  signalled.  Be 
sure  to  make  them  very  distinct  from  the  times  of  high 
concentration. 

Confidence  is  an  important  factor.  It  can  be  encouraged  by  ensuring 
that  trainees  feel  good  at  the  end  of  every  training  session.  Make 
an  effort  to  see  that  the  start  of  training  sessions  is  enjoyable.  This 
helps  trainees  come  into  practice  feeling  comfortable  and  self- 
assured.  Outright  intimidation  or  premature  challenges  may  have  a 
damaging  impact  on  confidence  and  self-efficacy,  even  if  the  trainee 
meets  the  challenge.  (Techniques  like  enactive  mastery  are 
applicable  here.) 

Although  much  training  may  not  be  intrinsically  enjoyable,  an  effort 
to  add  an  element  of  fun  may  prove  very  worthwhile.  A  positive 
mood  is  thought  to  be  fundamental  to  good  training  habits.  This 
does  not  mean  constant  laughing  and  smiling.  It  does  involve  a 
confident  outlook  on  both  life  and  training  and  can  be  caused  by  the 
feeling  that  one  is  doing  well.  For  this  reason,  reinforcement  and 
acknowledgment  of  progress  are  important.  Train  with  a  view  to 
establishing  a  positive  general  outlook,  and  desirable  training  habits 
and  good  results  should  follow  with  ease. 
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MOTIVATION 

People  who  want  to  learn,  learn  if  there  is  even  the  slightest  chance 
of  doing  so.  People  who  don't  want  to  learn,  generally  don't. 
Motivation,  therefore,  is  crucial. 

Motivation  varies  in  both  intensity  and  the  duration  and  reliability 
with  which  it  can  be  expected.  A  lot  ot  jobs  have  typical  motivation 
curves  -  inverted  bell  curves,  most  commonly,  or  in  some  cases  just 
the  descending  part  of  inverted  bell  curves!  Being  realistic  about 
learner  motivation  is  a  prerequisite  to  using  it  as  a  guide  to  training 
design  and  method  selection. 

A  high  level  of  motivation  has  to  be  present  to  achieve  excellence  in 
tasks  calling  for  significant  expertise.  The  learner  needs  to  be  able 
to  attend  consistently  to  the  most  difficult  parts  of  the  task,  that  is, 
to  spend  most  of  his  or  her  practice  time  doing  what  he  or  she  is 
worst  at.  Abundant  and  reliable  (mood-independent)  motivation  is 
necessary,  since  the  focus  is  consistently  on  the  least  obviously 
rewarding  parts  of  the  task. 

Most  tasks,  however,  do  not  require  that  level  of  motivation.  There 
are  a  few  simple  things  that  can  be  done  to  make  the  most  of  the 
available  motivation: 

•  Respect  it  and  and  don't  violate  it.  Training  should  both  be  and 
(a  different  thing)  show  itself  to  be  fastest,  straightest  way  to 
achieve  the  trainee's  goals. 

•  Reinforce  and/or  reward  progress  fairly.  If  motivation  is  viewed 
as  a  kind  of  exertion,  then  being  sure  that  training  includes 
events  with  unambiguous  beginnings,  middles,  and  ends  -  it’s 
much  easier  to  exert  when  you  can  see  the  finish  -  may  make 
maintaining  a  sense  of  progress  easier. 

•  Affirm  and  reinforce  belief  in  personal  control,  e.g.,  that  the 
learner  can  learn  and  master  the  material. 

•  Don't  over-challenge  early.  This  can  lead  to  discouragement  and 
a  reduction  in  belief  in  self-efficacy. 

•  Don't  under-challenge  late.  This  can  bracket  training  at  a  lower- 
than-necessary  level. 

Baldwin  et  al  (1991)  found  that  if  you  give  trainees  a  choice  about 
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what  and  how  they  want  to  be  trained,  it  raises  motivation. 
However,  soliciting  this  information  and  then  ignoring  it  is  a  violation 
of  trust,  and  is  measurably  worse  than  not  soliciting  it  at  all. 

A  sense  of  improvement  and  a  belief  that  the  learned  material  is 
useful  or  beneficial  is  fuel  to  motivation.  A  sense  of  statis, 
impossibility,  or  uselessness  is  poison.  Therefore,  be  realistic  about 
trainee  motivation  coming  in,  nourish  it,  and  avoid  squashing  it.  It's 
the  astute  trainer's  greatest  ally. 
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GOALS 


Goals  are  generally  helpful  in  training.  However,  there  are  different 
types  of  goals.  In  this  section,  we  distinguish  learning  and 
performance  goals. 

It  is  typical  in  training  to  state  a  goal  as  "be  able  to  perform  task  X, 
within  time  Y,  to  degree  Z"  -  measures  of  performance  (MOPs). 

These  are  performance  goals.  They  are  about  how  fast  and  how 
well  the  trainee  can  do  the  job. 

The  trainee's  job  is  to  learn  to  do  the  task,  and  it  is  typical  for 
trainees  to  interpret  their  job  to  themselves  as  a  matter  of  doing, 
rather  than  of  learning.  It  is  possible  for  trainees  to  have,  instead, 
learning  as  their  explicit,  conscious  goal.  Some  research  suggests 
that  trainees  who  approach  training  with  the  attitude  that  they 
want  to  improve,  much  more  importantly  than  doing  well  in  the  short 
run,  achieve  higher  levels  of  performance. 

To  begin,  we  will  contrast  the  effects  of  performance  and  learning 
goals.  This  is  a  biased  comparison— you'll  see  more  of  the  desirable 
effects  of  learning  goals  than  of  performance  goals.  With  that  in 
mind,  and  remembering  that  setting  learning  and  performance  goals 
is  NOT  an  either/or  proposition... 

Trainees  seek  to  maintain  positive  judgements  and  avoid  negative 
judgements  by  others,  by  seeking  to  show  or  document  their  ability 
to  satisfy  the  training  goal  -  or  by  seeking  to  avoid  discrediting  it.  To 
do  this,  they  focus  on  their  current  ability. 

They  are  concerned  with  measurement  of  the  ability  (MOPs),  asking, 
"Is  my  ability  adequate?"  When  they  practice,  they  do  it  to  find  out 
how  good  they  are,  to  test  themselves  against  perceived 
expectations,  and  to  judge  themselves  by  how  they  measure  up.  If 
they  don't  (which  they  probably  won't,  since  otherwise  they 
wouldn't  need  training),  they  can  experience  fear  of  failure, 
helplessness  in  the  face  of  failure,  negative  affect  (sometimes),  and 
impaired  performance  (things  go  from  bad  to  worse).  On  the  other 
hand,  if  they  meet  with  early  success  they  may  become  complacent 
or  overconfident. 
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The  problematic  aspects  of  performance  goals,  then,  are: 

•  Belief  in  low  ability  may  lead  to  loss  of  belief  in  utility  of  effort. 

•  Withdraw  of  effort  to  forestall  continued  documentation  of 
failure. 

•  Worry,  which  can  divert  attention  from  the  task  to  be  learned. 

•  Shame  and  anxiety,  which  motivate  escape  attempts. 

•  Trainees  can  self-handicap,  adjusting  their  level  of  effort  to 
maintain  an  "I  wasn't  really  trying"  margin. 

Any  or  all  of  these  can  lead  to  blockage  of  the  trainee's  crucial 
intrinsic  rewards  for  improvement. 

Learning  goals 

If,  however,  trainees  see  their  goal  as  learning,  they  will  focus  on 
increasing  their  abilities  and  mastering  new  tasks.  This  leads  to 
strategy  formation,  positive  affect,  and  sustained  performance. 

Instead  of  asking,  "Am  I  adequate?",  they  are  asking  "How  can  I 
best  acquire/improve  this  skill?"  Failure  becomes  valuable,  analysis 
of  failure  a  crucial  source  of  information— "If  knowledge  is  power, 
failure  is  fuel."  Failure  and  setbacks  lead  to  desire  for  more  practice, 
adjustments  of  strategy,  and  hypothesis  formulation  and  testing. 
The  inevitable  errors  are  not  perceived  as  failures  to  meet  goals  if 
the  trainees  are  focused  on  learning  goals. 

Learning  goals  can't  be  set  without  performance  goals— to  set 
sensible  learning  goals,  you  have  to  know  what  you're  trying  to  do, 
and  how  well  you  need  to  do  it.  Learning-as-goal  tends  more 
toward  attitude,  as  performance  goals  tend  more  toward  the 
concrete. 

To  facilitate  good  use  of  both,  the  trainer  can  set  up  a  distinction 
between  practice  and  performance.  Not  that  trainees  should 
practice  mistakes,  but  that  the  training  space  is  safe  for  mistakes. 
Trainees  are  not  punished  for  mistakes,  nor  are  training  sessions 
necessarily  used  as  tests  of  performance.  Then,  during  practice,  the 
trainees  are  focused  on  trying  to  get  better,  rather  than  to  perform 
at  a  particular  level  or  becoming  over-concerned  with  avoiding 
errors. 
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By  establishing  the  mindset  that  learning  is  the  most  important 
thing,  trainees  will  do  a  lot  more  learning.  This  will  lead  to  trainees 
who  are  less  inclined  to  give  up,  who  enjoy  training  more,  and  who 
will  choose  to  practice  to  learn,  instead  of  just  practicing  to  perform. 
It  will  also  lead  to  training  graduates  who  continue  to  improve  on 
the  job. 

That's  not  all  there  is  to  it,  of  course.  Just  as  with  performance 
goals,  learning  goals  need  to  be  set  at  the  right  level.  If  trainees  set 
themselves  unrealistic  learning  goals,  they  can  become  discouraged 
not  only  about  performance  but  about  their  ability  to  even  learn.  If 
the  goals  are  too  low,  again  the  problem  of  complacency  and 
plateau  may  arise. 

There's  one  more  thing.  The  trainer  can  suggest  learning  as  a  goal, 
but  if  he  or  she  suggests  particular  learning  goals,  these  transform 
themselves  into  performance  goals.  The  key  element  is  the  locus  of 
evaluation.  For  performance  goals,  it  is  external.  For  learning  goals, 
it  is  internal. 

Learning  goals  are  most  useful  when  you  are  trying  to  avoid  the  ill- 
effects  of  errors,  under  the  constraint  that  the  training  space  can  be 
made  safe  enough  for  errors.  There  are  many  variations  on  "safe 
enough"  in  this  context.  It  may  be  that  errors  are  easily  found  and 
corrected  (learning  computer  programming),  or  that  the  subject 
matter  can  be  altered  (medical  students  practicing  surgery  on 
cadavers).  The  word  "enough"  is  included  to  indicate  the  trainers 
responsibility  to  judge  both  safety,  and  to  the  degree  the  task  must 
be  altered  ("no  live  ammo"),  by  how  much  it  can  be  altered  and  still 
retain  its  essential  character. 

Learning  goals  are  relatively  more  important  the  more  the  trainee's 
ability  is  put  on  display  during  practice. 

For  simple  tasks,  or  for  trainees  who  believe  they  are  doing  great  all 
the  time,  performance  goals  alone  may  be  adequate. 

If  a  task  is  simple  enough,  or  draws  strongly  enough  on  existing 
skills,  e.g.,  just  reconfiguring  them  for  a  new  duty,  or  if  the  level  of 
performance  required  is  mild  enough,  learning  goals  may  not  be 
necessary. 
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If  the  task  is  found  by  the  trainees  to  be  intrinsically  interesting  - 
training  kids  to  play  video  games,  for  instance  -  they  will 
spontaneously  set  their  own  learning  goals,  and  trainer  intervention 
may  be  superfluous.  It  might  even  be  perceived  to  condescending. 

If  the  learning  is  never  fun,  or  the  emphasis  on  learning  makes  goal 
achievement  less  satisfying  (because  of  a  compulsion  to  move  to  the 
next  thing),  motivation  will  decline,  or  the  task  will  come  to  be  seen 
as  impossible.  Trying  to  improve  all  the  time  doesn't  really  leave  a 
lot  of  room  for  messing  around  and  having  fun,  which  in  some 
domains  -  science  and  engineering,  music,  athletics  -  is  an  important 
component  of  long-term  learning. 

In  sum,  it  is  rarely  wrong  to  set  learning  as  a  conscious  goal. 
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FEEDBACK 

Feedback  is  a  recurrent  theme  in  training,  but  there  are  quite  a  few 
things  covered  under  that  umbrella.  Information,  reinforcement,  and 
reward  and  punishment  are  all  called  by  that  name.  But  they're 
different  things.  We'll  briefly  try  to  pull  them  apart,  then  we'll  discuss 
feedback  in  a  little  more  detail.. 

Feedback  communicates  specific  knowledge,  typically  about  the 
difference  between  current  and  desired  performance.  If  it  comes 
from  a  trainer,  it  is  normally  given  to  improve  understanding. 
Reinforcement  generally  communicates  encouragement  (or 
discouragement),  and  is  given  for  motivation.  Reward  and 
punishment  are  also  given  for  motivation.  "Keep  your  elbow  high, 
it's  a  little  low..."  (information)  "...but  that  was  still  good!" 
(reinforcement)  "...  and  if  you  keep  it  up,  I'll  take  you  out  after 
practice"  (promise  of  reward). 

Because  attention  is  almost  invariably  a  form  of  reinforcement,  and 
because  feedback  is  invariably  a  kind  of  attention,  feedback  and 
reinforcement  are  tightly  intertwined.  However,  they  have  different 
goals  and  different  roles  in  training. 

Feedback  comes  in  two  forms,  self-directed  observation  and 
external  guidance.  For  each  of  the  two  kinds  of  feedback,  there  are 
two  forms.  Action  feedback  occurs  during  task  performance.  It  is 
most  effective  when  it  comes  in  direct  conjunction  with  the  action 
that  is  the  target  of  the  feedback.  Learning  feedback  occurs  after 
completion  of  task  performance. 

If  self-directed  observation  is  difficult  or  subtle  for  a  particular  task, 
there  are  two  options  for  training.  First,  the  environment  in  which 
the  task  is  performed  can  be  simplified  to  make  the  relevant  cues 
easier  to  recognize.  Second,  the  trainer  can  point  out  the  relevant 
cues  as  they  are  encountered  (external  guidance),  gradually 
reducing  the  feedback  as  the  trainees  become  accustomed  to 
noticing  and  correctly  interpreting  the  cues.  These  approaches  can 
be  used  either  separately  or  in  combination. 

Action  feedback  is  a  good  choice  when  (a)  a  single  element  of  the 
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task  is  the  subject,  (b)  the  trainees  are  able  to  absorb  and  use  the 
feedback  while  performing  the  task,  and  (c)  there  is  a  clear  path  to 
not  needing  the  feedback. 

Learning  feedback  is  appropriate  when  the  subject  is  interactions  of 
actions  taken  at  different  times  during  task  performance,  when  the 
topic  of  the  feedback  is  generalizes  over  multiple  actions,  or  when 
task  performance  is  so  demanding  that  trainees  cannot  really 
absorb  the  feedback  while  they  are  practicing. 

Holding  claims  that  action  feedback  leads  to  better  performance  in 
the  short  run  but  worse  long-term  learning.  The  gist  of  his  claim 
seems  to  be  that  if  trainees  use  action  feedback  "automatically," 
that  is,  just  do  what  they're  told  when  they're  told  to,  they  don't 
integrate  and  truly  learn  as  they  should.  Learning  feedback  can  t  be 
used  in  this  "just  follow  instructions"  way,  so  the  process  of  making 
sense  of  the  feedback  and  thinking  about  task  performance  leads  to 
better,  more  durable  learning. 

The  key  issue  is  the  degree  to  which  the  trainee  is  able  during 
practice  to  attend  to  his  or  her  intrinsic  cues  as  they  connect  with 
the  perceived  state  of  the  task  and  the  environment.  If  action 
feedback  is  internalized  and  not  just  followed,  then  its  immediacy 
can  strengthen  these  intrinsic  cues  and  speed  learning.  But  if  the 
task  is  too  fast-moving  or  complex,  such  attention  may  not  be 
possible  and  learning  feedback  will  be  better. 

For  some  tasks,  the  two  can  be  approximately  combined  by  using 
films  or  video,  so  that  the  specificity  of  action  feedback  can  be 
sustained,  but  the  attention  of  the  trainee  can  be  fully  on  learning 
(see  the  Guidebook  section  on  Behavioral  Modeling). 

Over  time,  less  and  less  action  feedback  should  be  necessary.  As 
this  occurs,  learning  feedback  can  shift  from  lower,  more  detail- 
oriented  feedback  ("always  try  to  keep  your  elbow  up  on  your 
forehand")  to  more  general,  "strategic"  observations  ("don't  let  him 
approach  the  net"). 

What  kinds  of  content  should  feedback  carry?  Essentially,  "what", 
"how",  and  "why". 
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What  includes  observations  of  various  specific  aspects  of  trainees' 
performance,  what  might  be  changed,  what  they  have  done,  the 
results  of  their  actions,  how  the  results  might  have  been  different, 
what  they  might  do  instead,  and  what  results  they  can  expect.  This 
can  also  include  providing  more  cues  for  the  trainee  -  timing  them, 
recording  them,  comparing  them  to  standards,  and  so  on.  This  may 
include  making  the  cues  stronger  early  in  the  training,  especially  if 
the  cues  are  very  hard  to  notice  unless  amplified.  The  trainer's  role 
here  is  to  increase  the  amount  of  information  available  to  the 
trainees. 

How  includes  helping  the  trainee  recognize  and  understand  the 
cues  provided  naturally  by  the  environment.  If  there  are  non- 
obvious  cues  that  have  a  particular  significance  these  should  be 
pointed  out.  Explain  the  relationship  between  the  environmental 
cues  and  the  trainees'  actions,  with  emphasis  on  cues  that  depend 
on  and  change  according  to  the  trainees'  actions.  Sometimes  it  is 
good  for  the  trainer  to  delay  or  even  withhold  feedback.  It  can  push 
the  trainees  toward  learning  to  look  for  and  use  direct  feedback 
from  the  source  (see  Feedback  -  Delays  and  Pauses  and  Feedback  - 
Withhold) 

Why  includes  instruction  on  the  goals  of  the  activity  and  how  these 
fit  into  a  longer-term  vision.  This  is  useful  especially  when  there  are 
many  ways  to  achieve  the  larger  goals  and  the  trainees  will  need  to 
think  independently.  It  will  also  reduce  reliance  on  the  trainer.  This 
in  turn  increases  the  trainees'  ability  to  improve,  since  the  trainee 
starts  to  become  his  or  her  own  trainer  (see  Advance  and  Back  Off). 

Common  problems  in  providing  feedback  are  pretty  predictable:  The 
wrong  amount,  the  wrong  content,  and  the  wrong  timing.  The 
amount  should  be  absorbable,  the  content  should  be  specific  and 
useable,  and  the  timing  should  respect  both  trainees'  receptiveness 
and  dignity. 

It  is  difficult  to  think  of  a  training  situation  where  feedback  is  not 
indicated.  The  training  design  choice  is  what  kind,  delivered  in  what 
ways,  at  what  times. 
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REINFORCEMENT 


In  the  previous  section  we  distinguisted  reinforcement  and  reward. 
Reinforcement  is  the  more  important  of  the  two. 

Reinforcement  is  a  response  that  occurs  concurrently  with  or 
immediately  after  an  action  is  taken.  Positive  reinforcement  is 
anything  that  increases  the  likelihood  of  the  action  being  repeated. 
Negative  reinforcement  is  anything  the  trainee  or  subject  wants  to 
avoid,  and  that  can  be  avoided  by  a  change  of  behavior.  There  are 
three  important  implications  of  this  definition: 

•  There  has  to  be  something  to  reinforce.  Reinforcement  only 
works  on  behavior  that  is  already  occurring  (even  if  it  is 
occurring  sporadically). 

•  Reinforcement  doesn't  have  to  be  pleasant  to  be  positive,  or 
unpleasant  to  be  negative.  Anything  that  increases  the 
likelihood  of  an  action  being  repeated  is  positive  reinforcement, 
and  anything  that  increases  the  likelihood  of  a  change  of 
behavior  a  negative  reinforcement.  Reinforcement  is  a  purely 
operational  concept. 

•  Reinforcement  is  different  than  reward  or  punishment.  Reward 
and  punishment  occur  after  (often  long  after)  the  actions  being 
rewarded  or  punished  have  been  taken.  And  negative 
reinforcement,  unlike  punishment,  can  be  "turned  off" 
immediately  by  a  change  in  behavior. 

Reinforcement  and  feedback  are  intertwined.  Reinforcement  is 
mostly  about  motivation,  feedback  about  changing  observable 
behavior.  However,  feedback  is  often  reinforcing  (negative  or 
positive)  and  reinforcement  certainly  changes  behavior.  When 
communicating  feedback,  the  trainer's  tone  of  voice,  physical 
attitude,  and  facial  expression  can  greatly  vary  the  degree  and  kind 
of  reinforcement.  Feedback  and  reinforcement  almost  always 
interact.  But  there  is  a  difference. 

As  there  are  two  kinds  of  reinforcement,  there  are  two  modes  for 
each  kind,  intrinsic  and  extrinsic. 

Intrinsic  reinforcement  is  when  you  feel  good  (or  bad)  inside 
because  of  what  you  have  done,  without  any  other  person 
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reinforcing  you.  This  is  the  simplest  kind  of  reinforcement.  You  draw 
a  picture,  and  you're  happy  doing  it. 

Extrinsic  reinforcement  comes  from  outside,  whenever  another 
person  reinforces  you.  That  it  is  another  person  is  important  to  the 
definition  of  "extrinsic".  If  an  action  has  a  direct  and  obvious  result, 
with  an  equally  obvious  desirability  of  outcome,  then  doing  well  or 
poorly  is  intrinsic  reinforcement,  even  though  the  stimulus  is 
external.  For  example,  in  basketball,  shooting  "free  throws" 
provides  immediate  reinforcement.  The  stimulus  is  external  -  you 
make  it,  or  you  don't  -  but  the  reinforcement  is  internal. 

Some  research  has  suggested  that  if  extrinsic  reinforcement  is 
applied  to  a  behavior  that  had  previously  been  motivated 
intrinsically,  then  when  the  extrinsic  reinforcement  is  subsequently 
removed,  intrinsic  motivation  to  perform  the  activity  drops  off.  In 
effect,  extrinsic  reinforcement  has  been  substituted  for  intrinsic,  and 
when  the  extrinsic  is  removed,  the  intrinsic  is  no  longer  there,  or  at 
least  has  been  significantly  dampened. 

This  is  a  concern  when  (a)  trainees  appear  to  be  overly  reliant  on 
the  trainer  for  reinforcement,  even  to  the  extent  that  they  really  just 
don't  want  to  do  it  without  reinforcement;  and  (b)  the  task  is  such 
that  it  is  reasonable  to  expect  it  to  be  intrinsically  reinforcing.  When 
this  is  true,  then  a  program  of  weaning  trainees  of  all  but  the  most 
important  reinforcement  is  called  for.  After  all,  by  doing  so,  you  are 
reinforcing  them  in  developing  their  own  internal  capacity  for  the 
task! 

The  research  on  whether  extrinsic  reinforcement  decreases  intrinsic 
motivation  is  controversial  and  intensely  debated,  though.  Because 
the  research  results  are  so  controversial,  not  reinforcing  people  to 
avoid  decreasing  intrinsic  motivation  should  be  done  with  care.  As 
long  as... 

•  The  learners  know  what  is  expected  from  them. 

•  The  trainer  provides  reinforcements  that  remain  sincere, 
meaningful,  and  appropriate. 

•  The  trainer  is  alert  to,  and  doesn't  allow,  trainees  to  become 
overly  reliant  on  the  extrinsic  reinforcements. 

•  The  learners  are  not  just  doing  the  minimum  necessary  to  get 
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the  reinforcement. 

...  then  extrinsic  rewards  are  great. 

Should  one  be  cautious  in  giving  out  reinforcement  because  it  might 
backfire?  Or  give  it  out  like  candy  (which,  in  some  cases,  it  might 
literally  be)? 

1.  Sometimes  people  only  work  for  the  reinforcement,  doing  the 
minimum  possible.  If  reinforcement  is  removed,  they  stop 
altogether.  But  these  effects  often  are  not  long  term,  and  the 
person  will  go  back  to  their  previous  level  without  the  reinforcer 
later. 

2.  If  the  intrinsic  reinforcement  is  weak  at  the  beginning,  extrinsic 
reinforcement  should  be  given  freely.  It  helps  to  get  people 
going,  and  can  make  a  difficult  task  seem  possible. 

3.  For  some  tasks,  the  work  is  not  ever  going  to  be  intrinsically 
reinforced.  Extrinsic  reinforcement  will  be  necessary  as  a 
structural  part  of  the  task. 

4.  The  trainer  needs  to  do  his  or  her  best  to  make  reinforcement 
meaningful,  precise,  and  well-timed.  People  detect  insincerity 
even  when  they  can't  consciously  put  their  finger  on  it.  In  one 
of  those  funny  twists,  the  more  reliable  and  generous  the 
trainer  is,  the  easier  it  is  for  trainees  to  give  up  reliance  on  the 
trainer's  reinforcement. 

Neither  intrinsic  nor  extrinsic  reinforcement  is  "better"  than  the 
other.  Humans  respond  to  a  mix,  and  the  particular  mix  should  be 
planned  according  to  the  nature  of  the  task  to  be  trained  and  the 
goals  of  the  training. 

Unintentional,  misdirected,  and  unusable  reinforcement  are  common 
problems.  The  first  occurs  either  when  the  trainer  is  responding 
without  being  conscious  of  doing  so  to  some  aspect  of  the  trainees' 
performance,  or  when  a  trainee  senses  an  unspoken  attitude  or 
reaction  from  the  trainer.  Misdirected  reinforcement  occurs  when  the 
trainer  simply  reinforces  the  wrong  thing,  whether  due  to  error  in 
timing,  beliefs  about  what  a  trainee  wants  as  a  reinforcer,  or  even  a 
misunderstanding  about  the  structure  of  the  task.  (The  trainer  isn't 
perfect.)  Unusable  reinforcement  can  occur  when  the  trainee 
perceives  himself  or  herself  as  doing  many  things  at  once,  and 
doesn't  know  what  aspect  of  his  or  her  performance  is  being 
reinforced;  or,  when  the  reinforcement  itself  is  vague— was  it 
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positive?  Negative?  Large?  Small? 

Reinforcement  is  such  a  general  approach,  and  there  are  enough 
circumstances  where  even  withholding  reinforcement  is  in  itself  a 
reinforcer,  that  the  only  case  where  reinforcement  won't  be  used  is 
probably  totally  unsupervised  task  immersion  ("push  'em  off  the 
dock  style  sink  or  swim").  Like  feedback,  the  question  is  not  whether 
to  use  reinforcement,  but  how  to  use  it. 
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CONSISTENCY  IN  REWARDS  AND  PRAISE 

Several  training  techniques  including  Attitude:  Locus  of  Control, 
Contract  Theory,  and  Socialization  involve  rewarding  improvement  or 
performance.  Praise  is  part  of  the  same  process  and  can  be  a  form 
of  reward  in  itself.  To  help  make  rewards  and  praise  effective  they 
must  be  consistent. 

The  basis  of  consistency  lies  in  a  clear  understanding  of  performance 
expectations,  assessment  of  trainee  capabilities,  and  knowledge  of 
previous  reward  levels  and  types.  Clear  performance  expectations 
mean  performance  itself  can  be  praised  and  rewarded  as  distinct 
from  improvement.  Awareness  of  trainee  ability  leads  to  rewards 
and  praise  when  improvement  is  shown  and  when  effort  is  made. 
Remembering  which  rewards  are  appropriate  for  particular  activities 
or  at  particular  times  helps  to  make  them  seem  fair  and  worthwhile. 

Avoid  unintentionally  rewarding  unwanted  behaviour  or  actions.  Use 
appropriate  training  techniques  to  lift  moods  or  disappointment.  Be 
aware  that  simply  paying  too  much  attention  to  negative  states  or 
activities  can  itself  constitute  reward  giving.  This  may  result  in  a 
repeat  of  the  negative  state  or  action  in  order  to  gain  the  "reward." 
The  best  approach  is  to  pay  a  lot  of  attention  throughout  all  the 
training  session.  It  is  important  also  not  to  let  moods  and  external 
circumstances  interfere  with  reward  giving.  If  praise  or  a  reward  has 
been  earned  it  should  be  given  even  the  trainer  does  not  feel  so 
inclined. 

Determining  when  to  give  rewards  for  effort  and  improvement 
should  be  based  on  what  is  reasonable  to  expect  from  a  trainee.  It 
should  also  be  calculated  to  produce  more  effort.  If  rewards  are 
delivered  too  frequently,  trainees  have  no  incentive  to  strive,  and 
the  effects  of  rewarding  will  be  diluted,  even  creating  a  loss  of 
credibility.  Rewarding  trainees  every  time  they  accomplish  a  simple 
task  will  not  be  productive.  A  reward  for  successive  accomplishments 
is  likely  to  produce  greater  effort.  Praise,  as  a  lesser  measure,  can 
be  used  as  encouragement  along  the  way. 

Remember  that  rewards  are  largely  considered  worthwhile  based  on 
the  credibility  of  the  reward  giver.  High  praise  or  large  reward  from 
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someone  ill-informed  about  the  subject  of  training  may  not  be  as 
effective  as  that  from  an  expert.  In  the  same  way,  praise  and 
reward  from  someone  considered  to  be  of  high  status  may  be 
valued  more  highly  than  from  someone  of  equal  or  lower  status  to 
the  trainee.  This  sort  of  factor  can  be  another  influence  when 
rewards  and  praise  are  assessed  for  consistency. 

Shaping  is  an  allied  technique,  but  has  the  aim  of  achieving 
successive  approximations  of  the  goal  which  is  different  from  the 
consistent  reward  of  effort.  Information  on  feedback  is  useful  for 
understanding  the  processes  at  work  when  rewards  and  praise  are 
used. 
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TRANSFER 


"Transfer"  is  what  shows  up  in  real-world  practice  as  a  result  of 
training.  The  goal  of  training  goal  should  be  for  the  trainees  to 
perform  better  out  in  the  world.  (Though  this  isn't  always 
remembered  as  well  as  it  should  be.)  The  basic  rule  of  transfer  from 
training  to  practice  is:  The  more  similar  the  training  is  to  the  real 
task,  the  more  complete  the  transfer. 

Unfortunately,  there's  a  trick  built  into  that  definition:  What  does 
"similar"  mean?  Superficial  characteristics?  Abstract  processes? 

Same  tools?  Similar  settings?  Internal  environment?  All  of  the  above, 
plus  something  else? 

The  significance  of  degree  of  similarity  is  keyed  on  the  importance  of 
the  particular  "feature"  of  the  training  situation  relative  to  actual 
task  performance.  A  "feature"  is  a  dimension  or  an  essential 
characteristic  of  domain. 

Baldwin  &  Ford  (1988)  say  that  transfer  happens  when: 

•  There  are  identical  stimulus  and  response  elements  in  training 
and  the  job. 

•  The  training  covers  general  principles  and  not  rote  practice. 

•  There  is  variability  in  the  problems  encountered  during  training. 

•  There  are  several  different  conditions/situations  for  practice. 

Noe  adds  that  transfer  is  likely  when: 

•  Trainees  are  confident  that  they  can  use  the  skills  correctly. 

•  Trainees  know  how  and  when  to  use  them  appropriately. 

•  Trainees  think  they  can  do  better  on  the  job  with  the  new  skills. 

Roulier  and  Goldstein  (1991)  classified  the  post-training  transfer 
environment  into: 

•  Situational  cues  of  three  kinds:  (a)  goals  that  serve  as 
reminders  to  use  the  training;  (b)  social  cues,  both  behavioral 
and  the  influence  of  peers  and  supervisors;  and,  (c)  task  and 
structure  cues,  that  is,  cues  from  the  job  itself. 

•  Consequences,  including  both  postive  and  negative  feedback, 
and  rewards  and  punishments. 

They  found  that  both  mattered  significantly. 
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Tannenbaum  and  Yukl  add  that  the  postraining  environment  can: 

•  Encourage  transfer  through  rewards  and  job  aids. 

•  Discourage  transfer  through  ridicule  from  peers. 

•  Inhibit  transfer  through  lack  of  equiptment  or  opportunity  to 
use  it. 

As  an  aside,  while  faithfully  reporting  their  results,  the  author  finds  it 
hard  to  get  excited  by  such  recommendations,  but  perhaps  they  are 
less  utterly  obvious  than  they  seem. 

Noe  stresses  the  importance  of  feedback  from  supervisors,  peers, 
and  subordinates.  You  have  to  have  a  positive  social  atmosphere  in 
which  to  transfer.  He  also  claims  that  it  might  help  to  provide 
organizational  rewards  to  people  who  provide  feedback  and 
encouragement  for  use  of  new  skills. 

In  summary,  the  fundamental  rule  of  transfer  is  to  make  the  training 
and  training  environment  as  similar  to  the  actual  task  environment 
as  possible. 

There  are  two  times  to  break  this  rule 

1.  When  exaggeration  can  aid  training,  e.g.,  faster  pace,  more 
intensity,  higher  loading,  greater  demand  for  precision,  etc.  Not 
only  can  this  increase  capacity,  it  can  make  actual  performance 
seem  easier  (because  it  is  easier). 

2.  When  more  variation  than  is  usually  seen  can  aid  training,  by 
supporting  generalization  and  distinction  between  the  task's 
varying  and  invariant  features. 
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SUMMARY  OF  UNIVERSAL  TRAINING  ISSUES 


We've  discussed  the  assessment  of  prerequisite  skills  and 
knowledge;  expectations;  motivation;  goals;  feedback; 
reinforcement;  and  transfer.  We  began  this  section  by  asserting 
that  none  of  these  were  problems  with  definitive  solutions,  an 
assertion  with  which  we  hope  to  have  satisfied  the  reader. 

We're  now  ready  to  consider  specific  training  methods  and 
approaches. 
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USING  THE  GUIDEBOOK 


In  this  section,  we  give  an  overview  of  ways  to  look  at  different 
kinds  of  training.  We  then  describe  how  to  locate  specific  kinds  of 
training  for  specific  tasks. 
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KINDS  OF  TRAINING  APPROACHES 

There  are  so  many  training  approaches  that  an  organizing 
perspective  is  helpful.  One  way  to  divide  training  approaches  is 
between  strategies  and  tactics.  These  are  points  on  a  continuum, 
which  are  easy  to  distinguish  in  the  extreme  cases,  e.g.,  structured 
state-space  differentials  as  a  strategy,  writing  on  the  blackboard  as 
a  presentation  tactic.  (Actually,  even  this  example  is  unclear- 
"writing  on  a  blackboard"  can  in  some  cases  be  a  strategy.) 

Strategies  can  be  roughly  divided  along  two  orthogonal  dimensions: 
telling/doing,  and  modeless/modal. 

Telling  strategies  involve  discourse  on  the  part  of  the  instructor  and 
reception  on  the  part  of  the  learner  -  that  is,  telling.  Telling  can 
include  verbal  and  textual  material,  diagrams,  pictures,  and  audio. 
Not  to  go  too  far  off  into  the  land  of  the  abstract,  but...  Halliday 
(1975)  categorizes  seven  language  functions.  He  call  four  pragmatic: 
Instrumental,  regulatory,  interactional,  personal.  Three  he  calls 
mathetic:  Heuristic,  imaginative,  and  informative.  Pragmatic  functions 
orient  and  act.  Mathetic  functions  create  knowledge  and  move 
beyond  the  immediately  referential.  Bruner  (1986)  points  out  that 
another  function  must  be  added,  the  metacognitive  capacity  to 
reflect  on  language  and  telling  as  a  subject  in  itself.  Telling  can  apply 
any  or  all  of  these  functions;  in  a  classroom  setting,  most  of  them 
will  be. 

"Doing"  strategies  involve  some  form  of  demonstration  or  direction 
on  the  part  of  the  instructor  and  active  imitative  or  goal-seeking 
behavior  on  the  part  of  the  learner.  Doing  can  involve  physical 
mimicry,  performing  experiments,  practicing  motor  skills,  using 
simulations,  or  constructing  artifacts.  Of  course,  it  can  also  involve 
telling;  for  example,  teachback,  in  which  the  learner  plays  the  role  of 
instructor,  is  a  form  of  doing. 

Telling  and  doing  are  meant  to  be  reasonably  broadly  construed.  It 
should  not  be  thought  that  telling  is  somehow  passive,  in  contrast 
to  doing,  which  is  dynamic.  Both  are  active.  A  Socratic  dialog  is  a 
form  of  telling.  The  distinction  is  that  in  telling,  internal  constructive 
action  is  required  of  the  learner,  while  in  doing,  both  internal  and 
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external  constructive  action  is  required. 

Modeless  strategies  present  material  in  the  "timeless  present,"  i.e., 
as  facts,  assertions,  or  operations  to  be  internalized  and  reliably 
reproduced.  Operationalizing  F  =  MA  with  respect  to  various  physics 
problems  is  an  example.  The  feeling  of  modeless  strategies  is  that 
the  material  is  somehow  out  there  in  the  world,  independent  of  any 
action  or  intention  of  the  learner. 

Modal  teaching  applies  stance  marking  (Feldman  &  Wertsch  1976), 
that  is,  use  of  linguistic  or  other  tokens  of  uncertainty  and 
probability  to  invite  negotiation  of,  and  responsibility  for,  meaning. 
"Stance  marking"  may  sound  like  what  dogs  do  to  fire  hydrants,  but 
it  is  a  real  thing.  Bruner  op  cit.  gives  the  example  of  his  fifth  grade 
teacher,  Miss  Orcutt,  making  the  statement  "It  is  a  very  puzzling 
thing  not  that  water  turns  to  ice  at  32°  Fahrenheit,  but  that  it 
should  change  from  a  liquid  into  a  solid."  Instead  of  taking  a 
modeless  approach  and  presenting  a  fact,  and  backing  it  by 
authority,  she  invited  her  students  to  share  her  fascination  with  the 
(rather  bizarre,  if  we  weren't  so  habituated)  phenomena  of  state 
transition.  Modeless  strategies  are  concerned  with  truth,  while 
modal  strategies  are  concerned  with  meaning.  Looked  at  from  a 
different  perspective,  modeless  strategies  rely  on  authority,  while 
modal  strategies  require  at  least  some  degree  of  learner  initiative. 

That  presentation  is  pretty  biased,  though.  It  makes  modeless 
strategies  sound  unimaginative,  authoritarian,  and  dull.  That's 
unfair.  Modeless  strategies  are  really  efficient,  and  sometimes  both 
teacher  and  learner  just  want  to  get  the  facts  across  in  as 
straightforward  a  way  as  possible.  If  all  you're  trying  to  do  is  tell 
some  trainees  where  the  on/off  switch  is,  you're  going  to  annoy 
them  if  you  invite  them  on  an  Imaginative  Journey  into  the  World  of 
Power  Switches. 

Organizational  strategies  are  slightly  simpler.  We  distinguish 
structural,  discovery,  and  narrative. 

Five  aspects  of  the  structural  strategy  (from  [Meyer  1975]): 

•  Covariance,  a  causal  relationship  between  antecedents  and 
consequences. 
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•  Comparison,  a  reference  to  known  structure(s),  either 
metaphorical  or  analogical. 

•  Collection,  clustering  related  concepts  or  operators,  in  time, 
space,  or  function. 

•  Description,  provision  or  general  statement  followed  by 
supporting  evidence. 

•  Response,  pairing  problem  and  solution,  question  and  answer, 
or  remark  and  reply. 

Discovery  techniques  can  be  "pure,"  guided,  or  expository.  In  pure 
discovery,  the  learner  is  set  in  the  learning  environment  to  find  on 
his  or  her  own  those  regularities  that  comprise  the  training 
objective.  In  guided  discovery,  the  learner  is  given  cues  to  focus 
search.  Both  pure  and  guided  discovery  are  inductive  approaches. 
Conventionally,  in  inductive  approaches,  the  target  rule,  concept,  or 
procedure  is  provided  after  the  learner  has  discovered  (induced)  the 
underlying  structure.  In  expository  discovery,  a  deductive  approach, 
the  rule  is  provided  at  the  start,  and  the  discovery  process  lies  in 
deductively  finding  its  correct  application. 

The  narrative  approach  provides  a  skeletal  plan  or  schema  which 
can  be  elaborated  and  refined.  Narrative  seems  the  most  flexible  of 
the  organizational  approaches. 

Presentation  tactics  for  instruction  include: 

•  Classroom  (verbal,  written,  diagrammatic,  pictorial/image, 
auditory). 

•  Structured  environment  (guided  and/or  discovery). 

•  Demonstration. 

•  Modeling  and  simulation  (real-world,  computer-based). 

•  Field  exercise  (guided  and/or  discovery). 

•  On-the-job  training  (OJT). 

•  Hybrid  approaches  such  as  field  class,  SAFOR,  or  classroom 
role-playing  (for  instance,  teachback). 
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THE  WEB  SURVEY 

To  use  the  automated  approach  to  generating  ideas  for  training 
approaches,  you  can  take  our  survey  and  we  will  make  some 
recommendations.  Following  are  the  statements  with  which  you  will 
be  asked,  in  textual  form.  For  each  statement,  you  will  be  asked  to 
register  a  judgement  of  the  degree  to  which  the  statement  is  true 
for  you  task. 

1.  The  task  to  be  trained  performed  in  a  physically  stressful 
environment. 

2.  The  task  to  be  trained  performed  in  a  physically  dangerous 
environment. 

3.  A  high  degree  of  physical  skill  is  required  for  the  task  to  be 
trained. 

4.  The  task  to  be  trained  needs  to  be  performed  when  fatigued. 

5.  There  are  large  variations  in  information  reliability  in  the  task  to 
be  trained. 

6.  There  is  a  high  degree  of  event  variability,  that  is,  big 
differences  between  minor  and  major  events,  in  the  task  to  be 
trained. 

7.  The  consequences  of  errors  are  very  serious  in  the  task  to  be 
trained?  ("Serious"  ranging  from  embarrassment  or  annoyance 
at  one  end  to  death  at  the  other.) 

8.  Degree  of  motivation  strongly  affects  performance  in  the  task  to 
be  trained. 

9.  Individual  autonomy  is  an  integral  part  the  task  to  be  trained. 

10.  Many  different  kinds  of  things  must  be  done  at  once  in  the  task 
to  be  trained. 

11.  The  task  to  be  trained  requires  extended  vigilance. 
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12.  Very  high  levels  of  expertise  can  be  achieved  in  the  task  to  be 
trained.  (Concert  musicians  and  neurosurgeons  are  examples  of 
high-expertise  practitioners.) 

13.  The  task  to  be  trained  requires  a  high  degree  of  intellectual 
skill.  (Scientific  research  is  an  example.) 

14.  Real-world  experience  is  vital  to  become  truly  qualified  for  the 
task  to  be  trained. 

15.  The  ability  to  explain  is  very  important  in  the  task  to  be  trained. 

16.  A  very  large  volume  of  factual  knowledge  is  needed  for  the  task 
to  be  trained. 

17.  A  very  large  volume  of  procedural  knowledge  is  needed  for  the 
task  to  be  trained. 

18.  Self-management  -  working  despite  stress,  concentrating 
despite  distraction,  maintaining  composure  -  is  critical  to  good 
performance  in  the  task  to  be  trained. 

19.  Mutual  monitoring  among  teammates  is  very  important  in  task 
performance. 

20.  Feedback  among  teammates  is  very  important  in  task 
performance. 

21.  Communication  among  teammates  is  very  important  in  task 
performance. 

22.  Reciprocal  backup  among  teammates  is  very  important  in  task 
performance. 

23.  Interdependence  among  teammates  is  very  important  in  task 
performance. 

24.  Predicting  teammates'  behavior  and  intentions  is  very  important 
in  task  performance. 
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25.  The  extent  to  which  a  teams  "jells"  is  very  important  to  task 
performance. 

26.  The  quality  of  leadership  has  a  very  strong  or  even  decisive 
effect  on  the  performance  of  the  team. 
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TRAINING  METHODS  BY  CATEGORY 


These  are  the  covered  training  methods,  arranged  in  categories. 
The  next  section  provides  an  alphabetical  list. 

Learning  physical  and  general  procedural  skills 

Adaptive  Training 

Automaticity 

Behavior  Modeling 

Break  Up  Practice 

Enactive  Mastery 

Establish  An  Environment 

Feedback  -  Withhold  It 

Feedback-  Delay  It 

Games  And  Simulations 

Learn  By  Doing  Only 

Let  Trainee  Become  Master 

Limit  Introduction 

Match  Instruction  And  Practice 

Mental  Simulation 

Overlearning 

Physical  Guidance 

Practice  Alone 

Shaping 

Memory 

Automaticity 
Behavior  Modeling 
Computer  Aided  Instructions 
Chunking 

Method  Of  Elaboration 
Method  Of  Loci 
Mold  Mental  Models 
Networking 

Planned  Cues  For  Memory 

PQ4R 

Recoding 

Visual  Cues  As  Prompts 
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Learning  complex  intellectual  material 

Automaticity 
Behavior  Modeling 
Computer  Aided  Instructions 
Deliberate  Practice 
Games  And  Simulations 
Let  Trainee  Become  Master 
Mental  Simulation 

Operationalize  Definitional  Concepts 

Overlearning 

Practice  Alone 

Self-Explanation 

Solve  Without  Examples 

Developing  high  levels  of  expertise 

Adaptive  Training 

Attitude  -  Locus  Of  Control 

Automaticity 

Deliberate  Practice 

Dont  Take  Your  Time-  Stress 

Enactive  Mastery 

Establish  An  Environment 

Games  And  Simulations 

Let  Trainee  Become  Master 

Mental  Simulation 

Mentors 

Overlearning 

Practice  Alone 

Self-Explanation 

Sequence  -  Hard  Or  Easy 

Shaping 

Solve  Without  Examples 

Dealing  with  stress 

Adaptive  Training 
Attitude  -  Locus  Of  Control 
Automaticity 

Dont  Take  Your  Time-  Stress 
Enactive  Mastery 
Establish  An  Environment 
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Games  And  Simulations 

Overlearning 

Stress  -  Criterion  Practice 

Stress  -  Emotional  Training 

Stress  -  Full  Information 

Stress  -  Graduated  Training 

Stress  -  Phased  Training 

Group  and  interpersonal  skills 

Behavior  Modeling 
Contract  Theory 
Establish  An  Environment 
Games  And  Simulations 
Mentors 

Paired  Post-Training 
Path-Goal  Theory 
Sequence  -  Hard  Or  Easy 
Shaping 

Stress  -  Emotional  Training 
Summarize  For  Class 
Teams  -  General  Methods 
Teams  -  Cooperative  Controversy 
Teams  -  Cross  Training 

Learning  when  the  actual  environment  can't  be  used 

Attitude  -  Locus  Of  Control 
Automaticity 

Computer  Aided  Instructions 
Establish  An  Environment 
Games  And  Simulations 
Mental  Simulation 
Overlearning 
Stress  -  Criterion  Practice 
Stress  -  Emotional  Training 
Stress  -  Full  Information 
Stress  -  Graduated  Training 
Stress  -  Phased  Training 

Long-term  learning  in  organizations 

Attitude  -  Locus  Of  Control 
Behavior  Modeling 
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Computer  Aided  Instructions 

Contract  Theory 

Deliberate  Practice 

Games  And  Simulations 

Making  Training  Seem  Important 

Mentors 

Paired  Post-Training 

Transfer 

Automaticity 
Behavior  Modeling 
Enactive  Mastery 
Establish  An  Environment 
Games  And  Simulations 
Mental  Simulation 
Paired  Post-Training 
Stress  -  Criterion  Practice 
Stress  -  Emotional  Training 
Stress  -  Full  Information 
Stress  -  Graduated  Training 
Stress  -  Phased  Training 
Transfer  -  Immediate  Practice 
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TRAINING  METHODS  ALPHABETIZED 

These  are  the  covered  training  methods,  as  an  alphabetical  list. 

Adaptive  Training 

Attitude  -  Locus  Of  Control 

Automaticity 

Behavior  Modeling 

Computer  Aided  Instructions 

Chunking 

Contract  Theory 

Deliberate  Practice 

Don't  Take  Your  Time-  Stress 

Enactive  Mastery 

Establish  An  Environment 

Feedback  -  Withhold  It 

Feedback-  Delay  It 

Games  And  Simulations 

Groups-  Cooperative  Learning 

Interactive  Imagery 

Learn  By  Doing  Only 

Let  Trainee  Become  Master 

Limit  Introduction 

Making  Training  Seem  Important 

Match  Instruction  And  Practice 

Mental  Simulation 

Mentors 

Method  Of  Elaboration 
Method  Of  Loci 
Mold  Mental  Models 
Networking 

Operationalize  Definitional  Concepts 

Overlearning 

Paired  Post-Training 

Path-Goal  Theory 

Physical  Guidance 

Planned  Cues  For  Memory 

PQ4R 

Practice  Alone 
Recoding 


Syukhtun  Research,  Inc. 


171 


Self-Explanation 

Sequence  -  Hard  Or  Easy 

Shaping 

Socialization 

Solve  Without  Examples 

Stress  -  Criterion  Practice 

Stress  -  Emotional  Training 

Stress  -  Full  Information 

Stress  -  Graduated  Training 

Stress  -  Phased  Training 

Summarize  For  Class 

Teams  -  General  Methods 

Teams  -  Cooperative  Controversy 

Teams  -  Cross  Training 

Timing  of  Practice 

Transfer  -  Immediate  Practice 

Visual  Cues  As  Prompts 

Visual  Cues  For  Navigation 
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ADAPTIVE  TRAINING 


Description 

Adaptive  training  makes  training  more  doable  for  trainees  by 
temporarily  making  the  task  easier,  without  breaking  it  into 
components. 

How  to  do  it 

As  its  name  suggests,  adaptive  training  is  the  process  of  adapting 
training  to  the  current  ability  level  of  your  trainees.  You  begin  with  a 
simplified  version  of  the  task.  As  trainees  gain  proficiency,  you  add 
complexity.  In  this  way,  you  step  them  up  until  they  are  doing  the 
actual  task. 

This  is  distinct  from  breaking  the  task  down  into  parts.  Here,  the 
trainees  are  doing  the  actual  task  with  the  actual  goals  throughout 
their  training.  The  task  is  simply  adjusted  so  that  it  is  at  the  right 
level  of  difficulty  for  the  trainees.  Maintaining  the  right  difficulty  level 
is  an  advantage  of  adaptive  training. 

If  trainees  need  to  learn  to  do  two  tasks  at  once,  you  need  to  assess 
the  interconnections  between  the  tasks.  If  they  are  loosely  bound, 
you  may  be  able  to  adjust  each  task  based  on  its  own  internal 
complexity.  If  the  tasks  are  tightly  interdependent,  you  may  have  to 
adjust  both  at  once,  each  relative  to  the  demands  of  the  other. 

Indicators 

This  provides  the  learner  with  a  task  that  is  not  too  difficult  to 
master,  when  the  actual  task  is  too  difficult  to  master.  This  approach 
is  especially  appropriate  when  the  task  is  not  easy  to  break  into 
components. 

Counterindicators 

Adaptive  training  is  an  alternative  to  breaking  the  task  into  subtasks. 
For  some  tasks,  it  is  better  to  teach  the  subtasks  and  their 
integration. 

When  using  adaptive  training,  you  need  to  be  careful  that  your 
trainees  don't  learn  to  do  the  task  wrongly,  or  learn  to  respond  to 
particular  stimuli  in  the  wrong  way,  which  they'll  have  to  unlearn  to 
perform  the  full  task. 
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It  is  also  true  for  this  technique  that  if  it  is  difficult  to  accurately 
gauge  how  well  trainees  are  doing,  then  it  will  be  even  more  difficult 
to  accurately  adapt  the  training  to  their  needs.  If  that's  the  case, 
looking  again  at  breaking  the  task  into  components  might  be 
worthwhile. 

Common  problems 

If  you  find  that  people  are  unable  to  keep  up,  don't  blame  the 
trainees  ("The  people  we  get  nowadays!  Why,  in  the  old  days...!") 
Make  the  task  easier.  But  if  they  are  bored,  make  it  harder,  because 
you  want  to  challenge  them.  It's  a  fine  line. 

If  you  find  that  the  student  is  not  really  ready  to  move  on,  but 
believes  that  he  or  she  is,  you  will  need  to  find  a  way  to  motivate 
them  to  continue  working  on  the  current  level  of  the  task  even 
though  they  think  they  don't  need  it.  This  can  be  hard  to  do. 

If  the  trainee  is  able  to  do  most  of  the  task  but  still  needs  work  on  a 
few  aspects,  if  possible  isolate  those  elements  and  work  on  them. 
This  is  really  component  training,  rather  than  adaptive.  However,  that 
may  be  appropriate  if  you  can  identify  specific  elements  that  in  fact 
can  be  isolated  and  practiced. 

Related  techniques 

Shaping 

Easy  first/ Practice  in  parts 
Deliberate  practice/Practice  in  parts 
Enactive  mastery 
Stress  -  how  to  practice 
Expectations  about  training 
Needs  assessment 
Stress— Phased  training 
Stress— Graduated  training 

References 

Holding 
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ATTITUDE:  LOCUS  OF  CONTROL 

Description 

This  approach  helps  trainees  to  understand  that  performance  and 
events  depend  on  what  they  do,  that  the  "locus  of  control"  lies  in 
themselves.  People  who  believe  success  results  from  their  actions 
achieve  better  results  than  those  who  tend  to  blame  failure  on  luck, 
fate,  other  people,  or  circumstance. 

How  to  do  it 

Simply  explain  the  desired  attitude  and  show  how  it  applies  to  the 
task  in  hand.  Identify  points  where  circumstance  might  be  blamed 
and  show  how  skill  and  concentration  are  key  factors.  Notice  and 
correct  inaccurate  attributions  to  luck. 

Create  an  environment  where  learners  are  coaxed  to  try  their  best. 
Leadership  is  important  to  show  the  natural  way  to  attribute  results 
is  to  oneself.  Realism  is  important,  though. 

This  technique  is  best  integrated  with  others,  including  socialization 
and  lead-by-example. 

Indicators 

You  can  use  this  approach  when  someone  is  making  a  lot  of  excuses 
and  or  not  taking  responsibility  for  what  they  do.  They  should  be 
encouraged  not  to  do  so,  and  especially  to  see  task-relevent 
instances  of  cause  and  effect  with  respect  to  their  actions. 

It  is  a  good,  basic  technique  to  influence  the  attitudes  of  your 
trainees,  or  to  help  set  up  a  positive  atmosphere  for  training. 

Counterindicators 

This  requires  fairly  high  trainer  skill,  and  content  may  not  always  be 
appropriate. 

Common  Problems 

Since  this  type  of  training  deals  with  personality,  you  may  find 
attitudes  are  ingrained.  Persistance  and  good  socialization  will  help 
to  overcome  this.  Allow  plenty  of  opportunity  to  identify  when  success 
is  due  to  individual  actions. 
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Related  techniques 

Socialization 

Goals  -  Learning  Vs  Performance 

References 

Noe 
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AUTOMATICITY 


Description 

Often  tasks  have  elements  that  can  be  done  automatically,  that  is,  as 
a  response  to  stimuli  without  having  to  think.  Learning  automaticity  is 
the  process  of  practicing  an  initially  conscious  action  or  sequence  until 
it  can  be  performed  without  conscious  attention.  Automatically, 
appropriately  applied,  not  only  leads  to  good  performance  of  the 
automatic  reaction,  but  also  frees  up  mental  resources  to  be  focused 
on  other  things. 

Automatic  tasks  are  often  quite  simple,  but  very  complex  behaviors 
can  become  automatic  -  such  as  driving  or  playing  some  kinds  of 
musical  passages. 

Automatic  responses  can  be  separated  two  groups.  One  is  where 
you  automatically  react  to  a  stimulus,  e.g.,  as  competitive  athletes 
must  in  some  circumstances.  The  second  is  where  an  automatic 
sequence  of  behaviors  is  followed  after  a  conscious  initiation,  e.g., 
tying  shoes.  The  elicitation  of  the  behavior  may  also  be  automatic, 
though  subject  to  normal  intentional  control,  but  once  the  process  is 
started,  it  runs  itself.  (If  the  training  goal  involves  sequence,  the 
section  on  Overlearning  may  also  be  applicable.) 

How  to  do  it 

There  a  a  number  of  approaches  to  training  automaticity. 

1.  Finding  things  to  automatize 

Use  places  where  novices  have  problems  and  experts  don't,  or  where 
novices  are  affected  by  stress  and  experts  aren't,  to  find  things  to 
train.  For  example,  if  experts  can  do  something  but  can't  tell  you  how 
to  do  it,  for  example  if  only  their  hands  know  how  to  do  it,  then  it  is 
likely  that  some  or  all  of  the  task  can  be  made  automatic. 

The  responses  don't  have  to  be  unique,  but  the  more  consistent  they 
are  (i.e.,  the  more  often  the  response  you  are  trying  to  train  is 
appropriate  to  the  stimulus)  the  easier  they  are  to  learn  and  the 
more  useful  the  training  will  be. 

2.  Repeated  practice 
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Automaticity  results  from  a  lot  a  practice.  The  trainees  must  be  able 
to  do  the  task  correctly.  Then  they  repeat  it  endlessly. 

If  there  are  components  to  the  task,  train  them  first,  then  integrate 
them.  The  way  that  some  complex  automatic  tasks  become  automatic 
is  that  you  learn  to  do  a  whole  bunch  of  simple  things  automatically. 

If  you  are  trying  to  train  someone  to  do  something  automatically,  you 
want  to  do  it  starting  with  simple  rules  and  reactions  that  they  can 
get  right,  and  then  make  it  more  complex  once  they  learn  the  simple 
material. 

3.  Point  out  relevant  cues 

Sometimes,  you  may  not  be  able  to  actually  tell  the  learner  any  useful 
rules  to  use.  If  not,  don't  give  up.  Just  identifying  the  stimuli  that  are 
predictive  may  help.  Let  their  brains  figure  out  the  rules  on  their  own. 
(It  may  be  that  you  don't  know  the  rules  because  your  brain  figured 
them  out  without  telling  you  what  they  were). 

For  example,  in  volleyball,  just  say  "I  want  you  to  really  look  at  their 
armswing  really  close  today,"  and  your  trainees  may  just  learn  to  pick 
up  important  cues  and  react  just  by  noticing  what  happens,  when 
without  focusing  on  it  they  might  never  have  noticed  it.  You  don't 
have  to  tell  them  the  actual  rules,  see,  you  just  tell  them  what  the 
antecedents  are  so  that  they  can  figure  the  rules  out. 

4.  Accuracy 

Do  not  sacrifice  accuracy  for  speed,  especially  in  the  beginning.  Speed 
will  develop.  In  fact,  do  not  sacrifice  accuracy  for  anything. 

Tannenbaum  and  Yukl  say  that  if  automatic  elements  of  the  task  are 
learned  incorrectly,  they  will  be  remembered  that  way  too.  Once  you 
learn  to  do  something  automatically,  you  stop  learning  unless  you  are 
forced  to  change  your  ways.  Much  better  to  learn  things  right  the  first 
time. 


5.  Do  not  change  the  response 

Kylonnen  and  Alluisi  point  out  that  it  is  a  very  bad  idea  to  train  people 
to  react  in  a  certain  way  to  a  cue  (automatically),  and  then  as  the 
learning  progresses,  make  them  change  the  way  they  react.  Avoid 
changing  requirements  midstream.  This  is  relevant  to  tasks  where 
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you  are  training  simpler  versions  first,  or  when  you  are  training  a  part 
of  a  task  that  the  person  is  inclined  to  do  differently  because  other 
factors  are  not  being  included. 

6.  Make  the  practice  environment  realistic. 

Present  the  stimulus  and  have  the  trainees  practice  in  a  realistic 
setting.  This  helps  with  generalization/transfer. 

If  you  can't  actually  present  the  cues  in  a  realistic  setting  (for 
example,  you  can't  light  a  person  on  fire  to  have  them  practice  "stop, 
drop  and  roll"),  then  other  routes  to  automaticity  must  be  pursued. 
Tell  them  what  to  do,  use  mnemonics,  or  use  mental  simulation  (have 
the  trainee  imagine  being  in  the  actual  situation  and  reacting 
appropriately). 

7.  Let  them  know  how  they're  doing 

Along  with  reinforcement,  giving  trainees  accurate  feedback  is  very 
important  for  training  people  to  do  things  automatically. 

8.  Increase  the  frequency  of  uncommon  events 

Once  basic  automaticity  is  achieved,  make  sure  that  uncommon,  but 
consistent,  events  are  experienced  often  enough  that  they  can  be 
automatized  -  by  making  them  occur  on  purpose,  speeding  things  up, 
or  other  changes  in  the  environment. 

9.  Complex  rules 

Sometimes  you  have  fairly  complex  rules.  For  example,  a  baseball 
player  does  not  have  time  to  figure  out  what  to  do  after  the  pitch  is 
thrown.  He  must  react  automatically.  But  baseball  players  use  the 
information  about  what  is  going  on  in  the  game  and  about  what  the 
ball  Is  doing  to  decide  whether  to  swing,  and  how  to  swing  (making 
sure  to  contact  the  ball  at  the  same  time).  The  rules  they  have  to 
follow  are  complicated  -  a  lot  of  things  influence  each  other,  the  cues 
are  not  easy  to  read,  and  it  is  a  hard  task  that  has  to  be  done 
automatically.  The  reasons  why  they  do  or  don't  swing  are  complex, 
but  are  based  on  some  set  of  automatic  rules  that  they  have  learned. 
If  you  are  forced  to  train  something  like  that  (heaven  forbid),  you  can 
still  train  it  automatically.  You  can  try  to  figure  out  individual  rules, 
and  implement  them  (e.g.,  "if  the  pitcher  twists  his  wrist,  expect  x"). 
On  the  other  hand,  you  can  also  train  the  person  to  do  it  without 
being  able  to  give  them  actual  rules:  You  can  just  give  them  feedback 
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and  instruction  and  let  them  learn  the  rules  subconsciously.  This 
doesn't  take  advantage  of  most  of  the  techniques  in  this  section,  but 
humans  are  good  at  automaticity. 

10.  Segmenting 

Train  the  hard  parts  of  a  complex  task  in  greater  depth  than  the  easy 
ones.  This  may  sound  like  the  opposite  of  automization.  It  is  often 
true  that  saying  something  is  "complex"  really  just  means  that  it  is 
hard  to  learn  -  either  the  set  of  rules  to  follow  based  on  the  cues  is 
complex  and  subtle,  or  because  the  behavior  is  hard.  Thus,  do  the 
hardest  thing  the  most. 

11.  Invent  inconsistency 

Training  them  in  less  than  perfectly  consistent  circumstances  may 
help  them  to  be  even  better  at  responding  once  you  put  them  in  a 
context  where  there  is  great  consistency.  So  if  you  have  them  have 
to  do  a  little  mental  processing  to  recognize  that  a  certain  response 
is  appropriate,  and  then  make  it  so  that  it  always  works,  their 
automatic  processes  will  be  especially  honed. 

This  is  best  to  do  after  they  have  learned  to  do  the  response 
automatically  and  correctly  in  the  first  place. 

12.  Train  with  a  timer 

Another  way  to  make  the  training  especially  strong  is  to  speed  it  up. 
For  one  thing,  this  forces  trainees  to  use  automatic  processes 
instead  of  relying  on  cognitive  resources.  You  might  even  speed 
things  up  beyond  the  speed  that  naturally  occurs  in  the  task.  People 
have  a  lot  of  potential.  If  they  can  learn  it  so  well  that  the  actual  task 
is  a  cinch  compared  to  the  training,  then  they  will  do  well  on  the  task. 
If  you  don't  challenge  them,  though,  that  potential  may  never  be 
realized,  because  people  tend  to  stop  improving  once  they  don't 
need  to  anymore. 

13.  Go  all  the  way 

If  there  is  something  you  can  identify  as  being  a  good  candidate  for 
automaticity,  make  sure  you  get  all  the  way  to  automaticity.  it  is 
worth  the  extra  effort. 

14.  Scripting 
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If  you  want  to  learn  to  do  a  sequence  of  behaviors  automatically,  you 
should  make  sure  that  each  step  in  the  sequence  automatically 
triggers  the  next,  so  that  you  don't  have  to  think  to  keep  the 
sequence  going. 

If  there  are  components  of  the  sequence  that  need  to  be  practiced 
on  their  own  for  the  trainee  to  be  able  to  do  them  right,  then  they 
should  be  practiced.  But  after  that,  the  trainee  should  just  do  the  full 
sequence,  over  and  over  again.  And  again  (see  overlearning).  And  to 
increase  transfer  and  generalization  they  should  do  it  in  different 
contexts,  both  circumstantial  and  temporal  (assuming  temporal 
generalization  is  a  desired  outcome). 

Indicators 

Tasks  that  include  standard  procedures  or  easy  elements  or  require 
repetition  can  be  automatized. 

Complex  behaviors  can  become  automatic,  if  they  simply  consist  of 
thorough  sets  of  rules  and  signals  and  procedures. 

In  situations  of  high  stress,  automatic  responses  and  scripting  will  be 
particularly  useful.  They  require  less  control  and  fewer  cognitive 
resources  -  meaning  that  (1)  they  are  harder  to  disrupt  by  stress, 
and  (2)  they  allow  you  to  think  about  other  things,  making  it  easier  to 
do  the  task. 

If  you  find  that  novices  find  something  very  hard  that  experts  can  do 
easily,  or  if  experts  can  do  it  but  can't  tell  you  how  to  do  it,  or  if 
experts  are  unaffected  by  stress  when  doing  something,  then  it  is  a 
good  bet  that  they  are  doing  the  task  automatically,  so  you  may 
want  to  try  to  train  the  novices  to  do  it  automatically  too. 

Counterindicators 

The  more  consistent  a  task  is,  the  better  a  candidate  it  is,  and 
conversely,  the  more  variable,  the  less  suitable.  If  every  time  you  see 
a  gun,  danger  is  present,  then  it  is  not  a  bad  idea  to  train  the 
automatic  response  of  alertness  and  preparation  to  flee  at  the  sight 
of  a  gun.  However,  if  every  time  you  see  a  knife  you  have  the  same 
reaction,  the  kitchen  is  going  to  become  an  overly  exciting  place.  You 
want  to  use  automatization  only  if  given  a  certain  cue  (or  set  of  cues) 
only  one  response  is  appropriate.  If  you  are  training  a  sequence  of 
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behaviors,  you  want  to  make  sure  that  once  the  sequence  is 
initiated,  it  is  only  supposed  to  go  one  way,  because  it  is  only  going 
to  go  one  way. 

A  drawback  is  that  this  approach  is  inflexible.  Automatic  responses 
are  likely  to  be  elicited  even  if  they  have  become  inappropriate.  More 
worrisome  still  is  lack  of  flexibility  when  there  is  a  sequence  of 
behaviors  (or  script)  that  you  have  learned.  If  the  set  of  steps 
doesn't  work,  you  are  going  to  do  them  anyway,  or  else  not  know 
what  to  do.  If  you  get  interrupted  in  the  middle,  you  may  have  to 
start  over,  just  like  tying  your  shoes.  If  this  is  not  a  problem,  then  it 
could  be  used  for  anything  where  there  is  a  single  pattern  that  is 
always  used  to  do  a  certain  task. 

Common  problems 

Do  they  do  the  same  thing  in  training  of  the  components  as  they  do 
when  they  are  actually  doing  the  task,  or  do  they  revert  to  some 
other  behavior?  Have  they  already  learned  to  do  things  wrong,  or  to 
ignore  certain  things?  Have  they  stopped  learning  new  responses 
and  started  just  getting  better  at  the  old  ones?  And  are  the  losing 
motivation  (training  the  automatic  parts  is  probably  going  to  be  fairly 
boring  and  not  get  them  spectacular  results  in  the  short  run).  Are 
they  learning  to  do  things  wrong  automatically? 

Related  techniques 

Overlearning 

Shaping 

References 
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BEHAVIORAL  MODELING 


Description 

Behavior  modeling  takes  advantage  of  the  human  ability  to  observe 
and  mimic.  The  trainer  performs  the  task  with  the  trainees  watching. 
The  trainees  then  imitate  the  trainer. 

Behavioral  modeling  should  be  preceded  by  an  explanation  of  what 
you  want  them  to  do  or  learn.  While  the  demonstration  is  being 
given,  it  may  be  appropriate  to  highlight  key  elements.  If  the 
demonstration  is  live,  it  may  be  useful  to  repeat  portions  of  the 
demonstration  while  the  trainees  practice. 

Behavioral  modeling  has  been  advocated  as  a  way  of  learning 
interpersonal  skills.  It  is  also  commonly  used  to  teach  physical  skills. 


How  to  do  it 

There  are  five  steps  in  behavior  modeling: 

1.  Modeling.  Present  a  demonstration  (live  or  on  tape)  of  a  skilled 
performer  showing  the  desired  behavior.  During  the  demo,  point 
out  the  important  elements,  things  to  be  learned,  how  it  all  fits 
together,  things  to  remember,  and  if  appropriate,  things  to  not 
learn. 

2.  Help  them  to  remember  and  understand  what  they  saw. 

3.  Have  the  trainees  practice.  The  more  the  better. 

4.  Give  them  feedback  on  their  practice,  including  (as  appropriate) 
repeated  exposures  to  the  modeling.  One  approach  to  this  is  to 
allow  trainees  to  watch  themselves  on  video,  with  expert 
commentary  reinforcing  correct  behavior. 

5.  Help  them  transfer  it  to  the  real  world. 

Behavior  modeling  is  a  good  technique  when  an  algorithmic  approach 
to  the  task  -  that  is,  an  approach  that  can  specify  each  step  -  is  not 
available.  Behavior  modeling  is  good  even  when  there  are  few 
concrete  guidelines  as  to  how  something  should  be  done,  and 
trainees  need  to  learn  skills  that  require  flexible,  adaptive  behavior. 
Behavior  modeling  should  be  used  if  for  the  particular  task,  it  is  simply 
easier  or  more  apt  to  demonstrate  than  to  tell.  This  is  certainly  true 
of  many  physical  skills,  and  in  domains  like  music  or  surgery,  verbal 
instruction  without  hearing  or  seeing  would  be  inadequate. 
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When  behavior  modeling  is  used  to  train  social  interaction  skills,  the 
trainees  should  be  exposed  to  the  general  principles,  see  these 
modeled  in  a  variety  of  different  situations,  and  then  be  given  the 
opportunity  to  practice  and  to  devise  alternative  ways  to  apply  the 
principles  in  various  situations. 

When  trainees  are  imitating  someone  skilled,  they  perform  better 
with  the  skilled  model  is  observing.  This  result  seems  like  it  might  be 
domain-sensitive,  though.  While  it's  easy  to  picture  for  athletes  being 
observed  by  their  coaches,  it's  harder  to  picture  managers  being 
more  effective  in  talking  with  employees  while  their  own  managers 
are  looking  on. 

It  may  be  hard  for  trainees  to  notice  or  to  pick  out  the  most  important 
elements.  It  is  a  good  idea  to  point  out  the  things  you  most  want  the 
trainees  to  see,  just  as  or  immediately  after  they  happen  in  the 
modeled  behavior.  This  is  especially  true  for  key  behaviors  that  are 
difficult  to  see  or  not  naturally  distinctive. 

The  "drop-step"  in  basketball  is  an  example.  Most  observers,  when 
shown  a  drop-step,  will  simply  see  the  player  with  the  ball  turning 
quickly  around  a  defender.  However,  an  essential  part  of  the  drop- 
step  is  to  be  positioned  correctly  at  the  initiation  of  the  move,  and  in 
making  the  turn  to  use  the  hips  and  butt  as  a  lever  to  force  the 
defender  away  from  the  direction  the  player  wishes  to  go.  This  use  of 
the  hips  must  be  pointed  out  and  explicitly  practiced. 

Without  articulated  learning  points,  there  can  be  problems.  For 
example,  one  study  followed  managers  who  were  taught  how  to 
reprimand  employees.  In  the  group  without  learning  points,  there 
was  marked  decrease  in  the  amount  of  positive  reinforcement  given 
out  by  the  trainees  when  they  returned  to  their  jobs.  The  model  had 
demonstrated  reprimanding,  but  what  the  trainees  apparently 
learned  was  "be  mean",  part  of  which  was  withholding  positive 
reinforcement  from  their  employees.  Learning  points  help  you  make 
sure  your  trainees  correctly  understand  what  you  are  trying  to  get 
across. 

Negative  learning  points  can  also  be  used.  If  there  is  only  a  single 
model  available,  he  or  she  may  unintentionally  add  idiosyncratic 
elements  to  the  demonstrated  performance.  If  this  is  the  case,  it  is 


Syukhtun  Research,  Inc. 


184 


appropriate  to  point  out  to  the  trainees  those  things  that  should  not 
be  imitated. 

As  long  as  the  appropriate  principles  are  being  followed,  let  the 
trainees  develop  their  own  rules.  They  will  come  up  with  rules  that 
are  not  necessarily  perfect,  that  may  be  inefficient,  but  they  will 
remember  them.  It  is  fine  to  suggest  rules  and  guidelines,  but  it  is 
important  to  allow  them  to  figure  out  their  own.  If  this  really  does  not 
seem  true  for  the  particular  task  to  be  trained,  then  behavior 
modeling  may  not  be  an  appropriate  training  technique. 

Behavior  modeling  relies  on  the  trainees  ability  to  observe,  imitate, 
and  self-correct.  The  more  accurately  trainees  can  observe 
themselves  and  compare  their  own  performance  with  the  models', 
the  more  quickly  they  will  be  able  to  advance.  Depending  on  the  task, 
any  of  mirrors,  audio  tape  recording,  videotape,  or  film  may  be  useful. 
Decker  found  that  combining  videos  of  trainee  performance  with 
expert  commentary  "enhanced  reproduction  skills"  (1983). 

Indicators 

If  it  is  harder  to  describe  how  to  do  something  than  it  is  to  show  it, 
behavior  modeling  is  appropriate.  And,  there  are  some  things  you 
just  can't  say.  Again,  behavior  modeling  is  called  for. 

Mills  and  Price  claim  that  behavior  modeling  works  best  for  skills  that 
are  concrete,  and  for  which  there  is  one  best  way  to  do  it,  with  little 
variation.  There  may  be  some  unintended  irony  here,  in  that  Mills  and 
Price  advocate  behavior  modeling  as  a  technique  for  training 
managers  in  social  interactions.  On  the  other  hand,  Tannenbaum  and 
Yukl  call  for  behavior  modeling  when  there  are  not  concrete 
guidelines  as  to  how  something  should  be  done,  and  trainees  need 
to  learn  skills  that  require  flexible,  adaptive  behavior.  These 
apparently  opposing  views  may  not  be  so  far  apart,  though,  since 
Mills  and  Price  were  considering  physical  skills,  while  Tannenbaum 
and  Yukl  studied  social  skills. 

Counterindicators 

If  it  is  possible  to  simply  explain  what  you  want  the  trainees  to  do, 
and  then  let  them  practice,  that  will  be  more  efficient  than  having 
them  try  to  pick  up  cues  from  a  person  modeling  the  behavior. 
Behavior  modeling  is  of  course  a  bad  choice  if  task  performance  is  not 
particularly  visible.  It  would  be  difficult  all  around  to  teach  math  with 
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behavior  modeling. 

Some  research  suggests  that  behavior  modeling  works  best  for 
trainees  with  moderate  to  high  self-efficacy.  Those  low  on  self  efficacy 
were  better  off  with  one-on-one  tutorial. 

Common  problems 

People  can  learn  any  number  of  things  that  you  do  not  intend  for 
them  to  learn.  Having  the  model  (or  better  yet,  different  models) 
perform  the  target  behavior  repeatedly  lets  the  learner  see  which 
elements  remain  constant  (and  are  therefore  the  important  ones.) 

Related  techniques  for  this  training  problem 

Physical  guidance 
Heuristics  -  Don't  take  your  time 
Lead  by  example  -  Behavior 
Lead  by  example  -  Attitude 
Feedback  -  Action  versus  learning 
Feedback  -  Delay  it 
Feedback  -  Withhold  it 
Mentors 

Teams  -  Cooperative  controversy 

References 
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COMPUTER  AIDED  INSTRUCTION 


Description 

There  is  an  ever  increasing  use  of  computers  in  instruction.  This 
section  has  some  information  about  when  computer  aided  instruction 
(CAI)  is  most  beneficial.  How  to  actually  design  and  build  computer- 
trainers  is  covered  in  other  sections.  This  one  tells  you  when  they 
may  be  useful  and  which  type  is  best. 

How  to  do  it 

Over  the  last  fifteen  years,  four  general  approaches  to  active 
computer-assisted  instruction  have  emerged.  (By  active  we  mean  to 
distinguish  these  approaches  from  simply  using  the  computer  as  a 
media  for  information  presentation.) 

The  first  to  emerge  was  model  tracing  [Anderson  et  al  1985].  Model 
tracing  assumes  that  there  is  a  right  way  to  perform  a  task  and 
attempts  to  determine  where  the  student  is  in  the  problem-solving 
sequence,  and  how  far  he  or  she  is  from  the  correct  protocol.  The 
advantage  of  this  approach  is  that  it  is  fairly  simple.  The 
disadvantage  is  that  it  puts  the  burden  on  the  student  to  figure  out 
how  and  why  he  or  she  was  on  the  wrong  track  and  how  to  return  to 
the  correct  approach. 

The  second  approach  is  reconstruction  [Naveh-Benjamin  et  al  1986] 
This  grew  out  of  an  awareness  of  the  problems  with  model  tracing. 
The  goal  of  reconstruction  is,  as  the  name  suggests,  an  attempt  to 
induce  a  model  of  what  the  student  must  have  been  thinking  to  have 
shown  the  observed  behavior  pattern,  that  is,  to  reconstruct  the 
students  model  of  the  domain  and  process.  If  model  tracing  is 
perhaps  too  simple,  reconstruction  is  too  complex.  In  very  tightly 
constrained  domains,  where  all  specific  behaviors  have  unique 
causes,  reconstruction  is  ideal.  But  for  the  great  majority  of  domains 
where  the  same  behavior  can  occur  for  many  different,  even 
unrelated  reasons,  reconstruction  flounders. 

In  response  to  this  problem,  issues  recognition  begins  by 
constructing  an  error  space  in  parallel  to  the  normal  desired 
performance  space  [Rodolitz  &  Clancey  1989].  It  assumes  that  there 
are  a  set  of  typical  failings  or  break  downs  in  understanding.  Then 
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the  user's  progress  is  monitored,  and  when  a  discrepancy  arises  the 
program  attempts  to  determine  which  typical  failing  is  present. 
Remediation  specific  to  that  particular  break  down  is  supplied. 

The  most  recent  and  successful  approach  is  our  choice  for  the 
proposed  work:  abstracted  problem  spaces,  which  are  actually  not  so 
abstract  as  they  may  sound.  The  basic  idea  is  simple.  In  any  problem 
solving  activity,  there  are  likely  to  be  conditional  steps  in  the  process 
of  solving  the  problem.  At  each  step,  certain  factors  are  relevant, 
others  are  not.  Some  kinds  of  data  are  germane,  other  data  not. 

Some  operations  are  applicable  and  useful,  others  not.  In  ad  di  tion, 
there  are  likely  to  be  some  markers  which  dis  tin  guish  each  particular 
stage  of  prob  lem  solving,  and  there  are  also  likely  to  be  markers  for 
valid  times  to  make  a  transition  to  another  problem  solving  step  (or 
space,  a  term  borrowed  from  mathematics).  The  process  of 
abstracted  problem  spaces  is  to  characterize  these  steps,  using 
cognitive  task  analysis  [Lesgold  et  al  1986],  and  then  to  construct  an 
environment  in  which  learners  can  realistically  practice,  exercise,  and 
experment  in  the  various  spaces. 

These  are  all  approaches  to  Al-based  CAI.  Most  CAI  is  based  more  on 
the  beliefs  and  tastes  of  the  designers  than  anything  else,  though. 
Not  surprizingly,  some  of  it  is  really  good,  and  some  Is  less  good. 

After  selecting  the  CAI  mmaterial,  provide  trainees  with  CD-ROM  and 
computer  equipment,  and  allot  a  time  for  completion  of  the  chosen 
material.  Some  studies  have  shown  that  a  CD-ROM  is  better  than 
lecture,  text,  videotape,  or  on  the  job  training,  though  this  result  is 
more  than  mildly  controversial.  It  has  been  shown  that  the  more 
interactive  the  CD,  the  better.  Note  that  low  variability  in  performance 
has  been  found  to  be  due  to  most  people  doing  well. 

Accountability  is  held  to  be  important.  For  this  reason,  it  is  probably 
best  to  be  prepared  to  follow  up  with  short,  but  comprehensive  quiz 
material  and  not  to  rely  entirely  on  interactive  correction  built  in  to 
the  program. 

Indicators 

Those  people  who  find  learning  difficult  -  not  people  who  find  the  task 
to  be  learned  hard  -  do  better  with  highly  structured  instruction, 
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broken  down  into  sequences  of  simple  units.  This  is  something  CAI 
delivers. 

This  method  of  training  is  easy  to  provide  since  it  relies  mainly  on 
equipment  alone,  yet  instruction  is  still  expert.  CAI  is  thought  to  be 
suitable  when  you  are  not  planning  a  lot  of  training,  when  the 
training  does  not  have  to  be  in  depth,  and  when  the  training  is  of  an 
exact,  set  nature. 

Counterindicators 

Successful  CAI  relies  on  the  choice  of  an  appropriate  CD-ROM. 
Considerable  investigation  may  be  required  to  find  this. 

CAI  is  not  usually  considered  helpful  when  training  needs  to  be 
flexible  or  customized  or  both. 

Common  problems 

The  CD-ROM  is  boring.  High  interactivity  is  considered  to  be  the  key 
factor.  Good  choice  is  essential  also  to  ensure  that  the  software  is 
pitched  at  the  right  level  of  difficulty.  It  is  also  important  that  CAI 
equipment  runs  at  adequate  speed. 

Related  techniques 

Games  and  simulations 

References 
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CHUNKING 


Description 

There  is  only  so  much  you  can  remember  at  one  time.  If  you  can 
group  different  pieces  of  information  into  packets,  there  will  be  less 
to  be  remembered  (without  losing  information). 

Chunking  is  normally  discussed  in  terms  of  the  aid  it  gives  to  short 
term  memory,  and  Kyllonen  and  Alluisi  are  no  exception.  However, 
they  do  indicate  that  it  helps  with  long-term  learning,  especially  in  the 
case  where  you  make  meaningful  chunks.  Newell  claims  that  chunking 
is  a  basic  human  learning  mode. 

How  to  do  it 

You  can  use  chunking  to  quickly  encode  or  memorize  things.  So  if  a 
phone  number  is  2300489,  one  thing  you  can  do  is  think  of  it  as  23, 
004,  89,  that  is,  three  numbers  instead  of  7. 

Another  approach  is  to  look  for  familiar  chunks  or  to  add  in  meaning. 
So,  004  could  be  like  James  Bond  007,  except  that  it  is  secret  agent 
Byron  Scott  (a  basketball  player  who  wore  the  number  4).  And  you 
can  also  remember  that  the  number  starts  with  the  first  ascending 
even-odd  single  digit  pair  (23),  and  ends  with  the  last  one,  (89). 

When  there  is  something  that  you  are  trying  to  keep  in  working 
memory,  or  if  you  want  to  learn  something  and  be  able  to  recall  it 
later  without  it  remaining  in  working  memory,  chunking  can  help, 
especially  if  you  form  meaningful  chunks. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 

References 
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CONTRACT  THEORY 

Description 

This  approach  involves  psychological  contracts.  An  understanding  is 
established  about  what  is  expected  of  the  trainee  and  what  he  or 
she  can  expect  in  return.  Agreement  is  reached  that  continued 
improvement  and  training  will  be  rewarded. 

How  to  do  it 

Before  training  starts  make  your  expectations  clear.  Outline  what 
actions  are  desired  and  ensure  that  rewards  or  advancements  are 
considered  fair,  meaningful,  and  desirable.  Rewards  can  simply  be 
related  to  the  activity  undertaken,  such  as  assignment  of  key  roles. 
Psychological  contracts  are  established  by  use.  Don't  say  it  if  you 
aren't  going  to  do  it. 

Be  consistent.  Contracts  of  this  sort  should  apply  in  the  long  as  well 
as  the  short  term.  Rewards  can  change  to  suit  progress  in  training, 
but  always  ensure  they  are  contingent  upon  clear  learning  goals. 
Improvement  should  be  the  major  factor  to  be  rewarded.  This 
accomodates  trainees  of  different  skill  levels. 

Indicators 

Contract  theory  can  work  as  a  specific  application  of  a  general 
management  promise  to  reward  training.  It  is  also  useful  in  any  long 
term  or  intensive  training  session.  It  may  also  serve  as  a  helpful 
approach  when  training  is  hard,  inherently  dull,  or  taking  place  in 
adverse  conditions. 

Counterindicators 

Contracts  may  not  help  when  training  is  very  short  and/or  easy,  or 
when  rewards  are  difficult  to  determine. 

Common  problems 

If  learning  goals  are  not  clear,  resentment  can  be  created  as  rewards 
are  not  forthcoming  when  expected.  Care  should  be  taken  to  avoid 
creating  favorites.  Keep  in  mind  that  improvement  is  more  important 
than  absolute  standards. 

Related  techniques 
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Making  training  seem  important 
Choices  about  training 
Establishing  a  positive  environment 

References 

Noe  et  ai 
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DELIBERATE  PRACTICE 


Description 

Deliberate  practice  is  is  the  process  of  isolating  the  elements  of  the 
target  task  most  in  need  of  practice,  and  practicing  them  methodically 
and  consistently  until  they  are  thoroughly  mastered.  Deliberate 
practice  is  iterative:  The  next  step  is  to  identify  the  next  element  of 
greatest  need,  again  to  be  mastered  through  concentration  and 
diligence. 

Deliberate  practice  is  highly  structured,  focusing  intensely  on 
improving  performance.  If  it  seems  redundant  to  ascribe  improvement 
to  a  training  approach,  remember  that  for  many  team  training 
approaches,  the  goals  include  intangibles  such  as  team-building, 
development  of  trust  and  appropriate  expectations,  sensitivity  to 
monitoring  and  signaling,  and  so  on.  In  contrast,  deliberate  practice 
is  concerned  with  observable,  tangible,  measurable  skills. 

There  may  be  social  and  team  tasks  for  which  deliberate  practice  is 
impractical,  or  not  sufficient  in  itself  to  achieve  mastery.  But  for 
individual  skills,  it  is  the  best  way  to  improve,  and  the  only  way  to  be 
the  best. 

How  to  do  it 

Six  preconditions  are  necessary  for  deliberate  practice: 

1.  A  well  defined  task.  The  nature  and  limits  of  the  task,  and  the 
criteria  for  good  performance  must  both  be  known.  More  than 
just  "known,"  they  should  be  clearly  apparent. 

2.  An  appropriate  level  of  difficulty.  The  task  must  be  difficult 
enough  to  justify  the  practice,  but  not  so  difficult  (or  so  hard  to 
decompose  into  practicable  chunks)  that  it  can't  be  approached. 

3.  Informative  feedback.  There  must  be  a  means  by  which  the 
practicer  can  clearly  discern  the  difference  between  current  and 
desired  performance.  The  more  immediate  the  feedback,  the 
better. 

4.  Opportunities  for  repetition  and  correction  of  errors.  The  practicer 
must  be  able  to  practice  the  hard  parts  frequently  and 
repetitively. 

5.  A  high  level  of  motivation.  The  learner  needs  to  be  able  to  attend 
consistently  to  the  most  difficult  parts  of  the  task,  in  effect  to 
spend  most  of  his  or  her  practice  time  doing  what  he  or  she  is 


Syukhtun  Research,  Inc. 


193 


worst  at.  An  abundant  fountain  of  motivation  is  necessary,  since 
the  the  focus  is  consistently  on  the  least  obviously  rewarding 
parts  of  the  task. 

6.  An  unflagging  willingness  to  exert  effort  to  improve.  Deliberate 
practice  is  hard  work,  and  must  be  consistent,  independent  of 
whether  the  learner  happens  to  be  enjoying  the  process  at  a 
given  time  or  not. 

Matching  the  six  preconditions,  there  are  six  steps. 

1.  Define  the  goals.  These  should  be  as  specific  as  possible,  and 
should  be  goals  that  can  be  accomplished  in  the  immediate  or 
near-term  future.  While  these  goals  should  fit  into  a  long-term 
progression,  the  current  step  is  to  define  the  immediate  goals. 

2.  Select  the  practice  targets,  which  may  require  decomposing  the 
task  into  practicable  sections. 

3.  Understand  the  performance  criteria.  Develop  clear  and  directly 
measurable  criteria  by  which  progress  and  performance  can  be 
judged. 

4.  Select  the  strategy  and  focus  for  practicing  the  task  section. 
"Strategy"  might  be  the  proportion  of  time  to  be  spent  on 
different  aspects  of  achieving  a  goal,  or  the  mix  of  isolation  and 
integration,  or  any  one  of  a  number  of  task  specific  things. 

"Focus"  would  be  the  emphasis  within  the  practice  chosen  to 
satisfy  the  performance  criteria,  whether  this  involved 
overcoming  a  weakness  or  refining  a  strength. 

5.  Establish  the  conditions  for  practice.  Where,  when,  and  for  how 
long.  "How  long"  is  discussed  below.  Practicing  alone  is  a 
frequently  mentioned  element.  One  study  found  that  differences 
between  musicians  in  time  spent  practicing  alone  correlated 
strongly  with  their  performance  levels,  but  that  total  time  spent 
playing  music  did  not. 

6.  Practice  the  right  amount,  concentrating,  until  mastery  is 
achieved.  The  "right  amount"  is  also  discussed  below. 
Concentration  is  the  key— when  you  can't  concentrate  any  more, 
stop!  Starkes  et  al  found  concentration  to  be  the  most  important 
element  of  deliberate  practice,  the  only  element  which  is 
absolutely  essential.  Galamain  and  Auer  found  that  when 
concentration  wains,  failure  to  stop  practicing  can  actually  lead  to 
a  decline  in  performance. 

To  make  some  of  this  more  concrete,  here's  a  worked  out  example. 
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First,  a  classical  guitarist  decides  that  developing  an  excellent  tremolo 
is  the  next  goal.  (A  tremolo  is  a  rapid  repetition  of  a  single  note,  or 
occasionally  of  two  notes.  Used  sparingly,  it  can  create  a  very 
beautiful  effect  on  the  guitar.)  Next  (Step  2),  the  guitarist  decides  to 
alternate  practicing  evenness  of  timing  and  of  volume,  concurrently 
alternating  between  completely  isolated  tremolos  and  one  or  more 
tremolo-dependent  songs.  Third,  the  guitarist  considers  the  criteria— 
evenness  of  both  volume  and  timing,  control  of  tone  color,  integration 
of  arbitrary  thumb  movements  with  the  tremolo  (which  is  typically 
played  with  the  first  through  third  fingers).  Fourth,  a  strategy  of 
beginning  with  tightly  focused  practice  on  timing  and  volume 
evenness,  then  intermingling  that  with  work  on  songs  that  exercise 
the  thumb  and  require  control  of  tone  color  and  musical  expression. 
The  focus  will  first  be  on  feeling  the  tremolo  in  the  fingers,  then 
achieving  automaticity,  where  the  hands  automatically  obey  the 
musical  intention. 

In  the  fifth  step,  the  guitarist  has  a  regular,  quiet  and 
infrequently  disturbed  practice  area  already,  and  decides  that  an 
hour  a  day  is  the  maximum  that  can  productively  be  spent  on  the 
tremolo  studies.  The  musician  is  in  the  habit  of  morning  practice,  and 
will  continue  with  that. 

Finally,  the  sixth  step.  The  practice  is  begun  and  pursued. 
Concentration  is  monitored  carefully,  since  the  simple  repetition 
required  to  perfect  the  tremolo  doesn't  provide  a  great  deal  of 
intrinsic  variety.  When  the  player  is  satisfied  that  temporal  and 
volume  evenness  has  been  achieved  at  a  wide  enough  range  of 
metronome  settings  (both  listening  while  playing  and  recording  may 
be  used  to  this  purpose),  songs  are  introduced:  Tarrega's  Recuerdos 
de  la  Alhambra,  Barrios'  Una  limosna  por  el  amor  de  Dios.  Again  using 
both  the  live  sound  of  the  playing  and  recordings,  the  guitarist  can 
assess  progress,  and  eventually  move  tremolo  practice  into 
"maintenance"  mode,  and  begin  the  cycle  again. 

Other  than  the  need  for  concentration  and  the  necessity  of  feedback, 
the  six  rules  given  above  are  guidelines,  not  laws.  For  example,  while 
deliberate  practice  is  normally  done  alone,  some  endeavors  simply 
require  the  presence  of  another  person.  Wrestlers  and  ballroom 
dancers  can  only  go  so  far  without  practice  partners.  The  partner 
provides  the  feedback  that  is  essential  to  improving  performance. 

Deliberate  practice  can  be  applied  to  any  skill  or  capacity  that  thrives 


on  practice.  Activities  like  long  distance  running  have  been  studied  as 
subjects  for  deliberate  practice.  Clearly,  compared  to  the  musical 
example  given  above,  running  is  not  a  highly  skilled  activity.  Training 
for  running,  however,  does  vary  greatly  in  skill,  with  some  runners 
being  much  more  successful  than  others  in  finding  the  fine  line 
between  training  for  maximum  performance  while  avoiding 
overtraining  and  injury.  The  interesting  point  is  that  deliberate 
practice  can  be  applied  to  something  where  the  performance  goal  is 
to  increase  a  capacity  rather  than  a  skill. 

How  long  to  practice ,  &  when 

Deliberate  practice  is  tiring.  Ericsson,  in  his  study  of  high  levels  of 
expertise,  found  that  experts  often  take  a  break  after  practicing  for 
about  an  hour,  and  they  rarely  do  more  than  about  four  hours  of 
deliberate  practice  a  day.  Any  more  and  exhaustion  starts  to  set  in. 
Ericsson  found  this  to  be  true  across  a  wide  range  of  occupations: 
Musicians,  athletes,  chess  players,  writers.  Expert's  actions  seem  to 
say  it  is  best  to  stop  once  concentration  fails. 

The  important  qualification  is  that  we're  talking  about  deliberate 
practice,  not  total  time  on  task.  A  musician  may  play  for  ten  hours  a 
day,  but  they  tend  to  do  deliberate  practice  for  no  more  than  about 
four.  The  rest  of  the  time  may  be  spent  on  material  that  is  already 
well-mastered,  or  in  free  improvisation,  or  in  any  number  of  ways.  But 
the  focus,  concentration,  and  intense  attention  to  feedback  will  not 
be  the  same. 

Gradual  increases  in  amount  of  deliberate  practice  over  time  are  the 
norm,  probably  as  the  capacity  to  concentrate  on  the  particular  task 
develops.  Being  able  to  do  four  hours  of  intense  practice  a  day  is  an 
achievement  in  itself.  It  comes  with  a  price,  of  course.  Four  hours  of 
deliberate  practice  can  be  exhausting,  and  experts  tend  both  to 
sleep  and  to  take  naps  more  than  non-experts. 

Experts  in  tasks  requiring  intellectual  skills  most  frequently  practice  in 
the  morning,  when  capacity  for  complex  cognitive  effort  is  greatest. 

On  the  other  hand,  athletes  tend  to  train  in  the  afternoons  or 
evenings,  when  perceptual-motor  abilities  are  at  their  best. 

Just  practicing  a  lot  is  not  a  substitute  for  deliberate  practice.  It  is 
possible  to  practice  and  not  improve,  and  some  kinds  of  practice  are 
clearly  better  than  others.  Ericsson  makes  the  controversial  claim  that 
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when  a  number  of  people  do  deliberate  practice,  all  will  learn  about 
the  same  amount  if  they  practice  the  same  amount  of  time.  Stated 
another  way,  the  claims  is  that  some  people  are  not  more  talented 
than  others;  what  makes  perfect  is  not  talent,  but  practice.  This 
assertion  will  be  examined  below. 

If  you  want  to  be  the  best,  you'd  better  be  ready  for  a  long 
period  of  gradual  improvement,  because  to  achieve  very  high  levels 
of  accomplishment  seems  to  require,  in  general,  at  least  10  years  of 
pursuit. 

Motivation 

Deliberate  practice  is,  in  general,  not  fun.  As  mentioned  above,  it  is 
often  a  commitment  to  focus  on  what  the  learner  is  worst  at.  That's 
why  it  is  so  hard,  and  why  most  people  don't  do  it. 

The  motivation  for  doing  deliberate  practice  is  that  in  the  long  term  it 
makes  you  better.  Short-term  rewards  must  often  be  set  aside. 

When  talent  is  even  roughly  equal,  coaches  and  experts  agree  that 
the  most  important  factor  in  eventual  success  is  desire  to  succeed, 
the  motivation  that  supports  the  learner  through  all  the  intense  work 
of  deliberately  practicing. 

It  doesn't  have  to  be  grim,  though.  In  order  to  ease  motivation, 
stressing  the  likelihood  of  future  rewards,  inventing  short  term 
rewards,  and  embedding  learning  in  enjoyable  situations  can  all  be 
productively  used 

Ultimately  though,  for  the  task  to  truly  be  mastered,  the  learner  has 
to  take  a  deep  pleasure  in  its  performance.  In  a  Kenneth  Cooper- 
directed  study  contrasting  world-class  and  national-class  male 
runners,  it  was  found  that  the  national-class  runners  actually  trained 
more  than  the  world-class  runners.  (Don't  get  the  wrong  impression 
that  the  world  class  runners  did  it  all  on  talent;  it  appeared  rather 
that  the  national-class  runners  were  simply  more  likely  to  overtrain.) 
The  most  interesting  finding  relative  to  motivation  was  this:  The 
world-class  runners  generally  reported  their  primary  motivation  was 
that  they  loved  to  run.  They  reported  that  they  would  run  whether 
they  raced  or  not.  The  national-class  runners,  in  contrast,  reported 
themselves  motivated  much  more  by  desired  racing  success.  Their 
"love  of  running"  rating  was  much  lower.  Deliberate  practice  is  simply 
too  hard  to  do  if  you  don't  at  a  deep  level  truly  enjoy  the  process. 
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Natural  ability 

When  talking  about  talent  or  natural  ability,  expertise  researchers 
generally  claim  that,  yes,  some  people  are  talented,  but  that  what 
"talent"  really  is,  is  the  person's  natural  predisposition  (his  or  her 
attitude,  emotions,  motivation)  toward  doing  deliberate  practice. 

They  claim  there  is  little  or  no  role  for  natural  ability  for  a  specific  task, 
except  insofar  as  the  task  requires  an  obvious  physical  characteristic 
such  as  height  or  bulk. 

For  example,  Ericsson  cites  Polgar,  who  decided,  with  no  a  priori 
reason  to  think  that  his  daughters  would  be  especially  talented,  that 
as  an  educational  experiment  he  would  train  them  as  chess  players. 
His  technique  was  early  and  focused  training— deliberate  practice. 

The  three  sisters  went  on  to  become  first,  second,  and  sixth  ranked 
women  in  the  world.  The  youngest  daughter  became  the  youngest 
grandmaster  chess  has  yet  seen. 

Despite  such  compelling  anecdotes,  the  author  believes  the  claim 
that  natural  talent  has  little  meaning  is  an  artifact  of  the  researcher's 
study  designs.  Typically,  researchers  have  studied  what  distinguishes 
the  best  experts  from  their  high-level  peers.  Why  is  someone  the 
Symphony's  First  Violin,  and  someone  else  Second?  What  makes  this 
player  an  NBA  starter,  and  this  other  a  back-up?  But  this  is  surely  a 
skewed  sort  of  sample.  It  doesn't  begin  with  a  representative  cross- 
section  of  aspirants  (high  school  freshman  basketball  players,  for 
example,  even  though  that  is  already  a  skewed  sample).  Rather,  it 
begins  after  many  years  of  winnowing  and  selection  have  taken 
place.  Beginning  with  the  highly  talented  and  looking  for  variation 
assumes  talent  as  a  constant.  This  may  be  fine  for  distinguishing 
among  high  levels  of  expertise,  but  it  doesn't  justify  the  claim  that 
talent  doesn't  count. 

If  "obvious"  elements  like  height  can  be  allowed  as  talent  that  does 
count,  what  principled  way  is  there  to  filter  out  the  slightly  less 
obvious?  Cardiovascular  capacity  and  maximum  oxygen  uptake,  for 
example?  Reflex  speed?  Mix  of  fast-twitch  and  slow-twitch  muscle 
fibers?  Balance  and  coordination?  For  that  matter,  working  memory 
capacity?  It  is  a  strained  argument  to  say  that  the  things  that  are 
easy  to  see  can  be  allowed  as  important  natural  talent,  but  things 
that  are  harder  to  see  or  quantify  can't. 
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Limits  of  deliberate  practice 

The  amount  of  deliberate  practice  seems  to  be  what  separates  the 
best  experts  from  their  peers.  Deliberate  practice  is  tremendously 
powerful,  but  it  is  not  always  enough. 

•  Tasks  vary  in  the  degree  to  which  they  can  be  broken  into 
components,  and  in  the  degree  of  difficulty  involved  in  integrating 
separately  practiced  components. 

•  The  degree  of  control  over  practice  conditions  can  vary  greatly 
from  task  to  task.  It  is  easier  for  musicians  and  chess  players  to 
control  the  scheduling  and  the  material  of  their  practice  than  it  is 
for  surgeons. 

•  Repeatability  is  variable  among  tasks,  especially  when  multiple 
learners  are  involved,  or  competition  is  significant.  If  the  task 
requires  interaction  with  a  dynamic  environment,  repeatability 
decreases  as  well. 

•  Feedback  also  varies  in  its  immediacy  and  its  reliability.  For 
example,  experienced  distance  runners  know  that  the  most  likely 
time  to  sustain  an  injury  is  when  feeling  great  (because  that's 
when  overextension  is  most  tempting). 

Finally,  there  is  the  fun  factor.  Deliberate  practice  is  described  as 
being  "fun-independent",  something  done  regardless  of  pleasure  or 
the  lack  thereof.  But  in  Starkes  et  al's  study  of  top  athletes,  things 
rated  as  most  fun  to  do  were  also  rated  as  most  important  for 
training.  Some  things  critical  for  improvement  are  fun  (if  hard),  even  ' 
when  they  don't  allow  for  a  great  deal  of  precise  repetition  and 
feedback.  Basketball  players  scrimmage.  Jazz  musicians  jam. 
Philosophers  argue.  These  are  all  fun  for  the  participants,  necessary 
for  improvement,  require  concentration,  but  each  fails  the  criteria  of 
deliberate  practice. 

Because  amount  of  deliberate  practice  consistently  seems  to 
distinguish  the  best  experts  from  their  (marginally)  less-accomplished 
peers,  it  is  tempting  to  say  that  deliberate  practice  is  all.  However, 
this  does  not  mean  that  other  things  are  not  important.  Even 
something  as  trivial-appearing  as  "messing  around"  during  practice  of 
a  sport  or  skill.  If  all  of  the  best  practitioners  mess  around  at  least 
enough,  then  that  will  not  show  up  in  the  statistical  analyses  used  to 
distinguish  levels  of  expertise.  As  argued  above,  that  it  doesn't  show 
up  doesn't  mean  it  is  unnecessary. 
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Why  doesn't  everybody  do  it? 

Deliberate  practice  may  yield  less  impressive  results  In  the  short  run, 
and  It's  hard  and  it  often  isn't  fun.  Whatever  disagreement  there  may 
be  about  other  aspects  of  natural  talent,  everyone  agrees  on  the 
need  for  determination  that  Is  well  above  average.  It's  said  that 
when  Magic  Johnson  was  a  kid,  he'd  practice  in  weather  so  cold  his 
basketball  would  break. 

Indicators  . , 

Deliberate  practice  Is  Indicated  when  learners  want  to  achieve  very 
high  levels  of  proficiency,  are  willing  to  put  a  lot  of  effort  into  training, 
and  expect  to  practice  the  skill(s)  for  a  prolonged  period  of  time- 
meaning  continually  and  indefinitely.  The  skills  most  appropriate  for 
deliberate  practice  are  those  whose  perfection  never  ceases. 

Counterindicators 

Difficult-to-decompose  task  structures,  high  degrees  of  variation  In 
task  conditions  or  performance,  difficulty  of  controlling  practice 
circumstances,  problematic  feedback,  and  low  repeatability  all 
mitigate  away  from  deliberate  practice. 

Common  problems 

Loss  of  motivation  can  occur,  especially  as  a  result  of  consistent  focus 
on  the  weakest  elements  or  performance,  and  an  accompanying 
perception  of  slow  progress.  Having  a  range  of  objective,  external 
performance  benchmarks  can  sometimes  ameliorate  this. 

Monitoring  concentration  during  practice  is  an  important  skill,  and 
being  willing  to  stop  when  it  is  the  right  time  takes  discipline. 

There  Is  a  natural  tendency  to  want  to  perform,  or  to  become 
satisfied  with  what  has  been  accomplished.  Paradoxically  (since 
deliberate  practice  focuses  on  performance),  the  more  the  attention 
remains  on  learning  goals,  in  contrast  with  performance  goals,  the 
more  effective  deliberate  practice  will  be. 

Related  techniques 

Learning  versus  Performance  Goals 

Automatlcity 

Overlearning 

Feedback  Sections 

Goal  Sections 
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Sequence  -  Hard  Or  Easy 

References 

Ericsson 
Starkes  et  a  I 
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DON’T  TAKE  YOUR  TIME: 
DECISIONS  UNDER  STRESS 


Description  . 

Under  stress,  focus  of  attention  narrows,  and  decision-making 

strategies  that  are  overly  complex  fail.  In  stressed  sltuffclJ"®  tJeJf 
may  be  neither  time  nor  attention  to  employ  complex  strategies,  it 
makes  sense,  then,  that  people  should  not  be  trained  to  use  a 
complex  strategy  If  they  won't  be  able  to  use  it  in  the  ac  u 
situation. 


Heuristics  that  minimize  demands  on  mental  resources  are  much  less 
vulnerable  to  stress  than  more  analytic  and  time-consuming 
strategies,  even  though  the  latter  may  be  more  accurate.  Because 
heuristics  are  simpler  and  more  distilled  by  experience,  even  a 
narrowed  focus  can  include  more  of  them  than  tt  can  somet  ng  more 
complex.  Second,  automatic  and  overlearned  responses  are  more 
likely  to  be  fully  Integrated  with  heuristics,  which  both  reduces  the 
effects  of  time  and  other  stressors  on  performance,  and  frees 
(heuristic-guided)  attention  for  the  areas  where  It  Is  most  likely  to  oe 

valuable. 


Train  your  people  with  decision-making  tools  they  will  be  able  to  use 
in  the  situations  they'll  actually  have  to  use  them  in. 


How  to  do  it  t 

Don't  train  complex  decision  processes  if  these  cannot  be  used  on 
the  real-world  task.  Your  trainees  will  do  worse  than  If  you  dian  t 
train  them  at  all.  Instead,  let  them  know  what  outcomes  are 
expected,  then  give  them  the  opportunity  to  practice  In  a  variety  or 
situations,  with  an  Increasingly  realistic  stress  loading. 

Practice  situations  should  be  set  at  the  "right"  tempo  and  stress 
levels.  It  is  a  mistake  to  put  trainees  in  situations  where  they  just 
can't  keep  up.  It  destroys  confidence  and  leads  to  learning 
Inappropriate  adaptations.  Slowing  the  training  task  down  or 
reducing  the  situational  stress  may  be  called  for,  especially  If  tnere 
are  important  elements  of  the  task  or  the  task  environment  they 
need  to  become  accustomed  to.  They  should  be  allowed  to  learn 
these  In  an  easier  or  slower  (or  both)  setting  than  they  will  later 
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encounter  In  real  life.  However,  caution  should  be  exercised  to  avoid 
letting  trainees  practice  and  become  comfortable  doing  the  task 
slowly  or  In  too  easy  an  environment.  Speed  and  stress  should 
always  be  great  enough  that  the  trainees  use  the  same  decision 
strategies  they  will  use  In  the  criterion  situation. 

If  there  are  prototypical  situations  they  will  encounter  over  and  over, 
provide  them  with  a  repertoire  of  known,  good  decisions  and  the 
opportunity  to  practice  these  in  a  realistic  range  of  situations.  If  this 
is  not  true,  or  If  they  will  need  to  create  the  desired  outcomes  in  a 
very  wide  range  of  circumstance,  then  insofar  as  possible,  let  them 
develop  their  own  heuristics.  As  long  as  the  trainees  receive 
adequate  feedback  during  training,  their  own  heuristics  will  be  more 
deep-seated,  and  therefore  more  reliable  under  intense  stress,  than 
imposed  rules. 

On  the  other  hand,  if  a  particular  structure  for  decision-making  must 
be  used,  finding  mnemonics  or  heuristics  that  lead  to  the  right  set  of 
criteria  can  ease  the  learning  and  recall  tasks.  In  the  U.5.  military,  for 
example,  "METT-T"  Is  the  standard  mnemonic  for  leaders  In  the  field 
Mission,  Enemy,  Terrain,  Troops,  and  Time.  Not  a  decision  process, 
per  se,  but  a  quick  way  for  commanders  to  make  sure  they  have 
considered  the  most  important  relevant  factors  in  their  highly 
stressed  field  decision  making. 

If  the  task  is  decomposed  for  teaching,  care  should  be  taken  that  the 
divisions  are  as  "natural"  as  possible  with  respect  to  to  how  the 
trainees  will  need  to  perform  in  the  real  world.  Learning  artificial 
components  can  lead  the  trainees  to  develop  heuristics  or  decision 
approaches  in  training  that  are  "good"  for  training  but  Inadequate  for 
real  life. 

Finally,  let  the  trainees  know  that  they  need  to  practice  monitoring 
their  own  decision  processes.  In  order  to  make  the  best  decisions 
they  can  under  stress,  they  need  to  learn  to  watch  themselves  so 
that  they  use  the  right  amount  of  time  to  decide  (nether  too  much 
nor  too  little),  that  they  don't  fixate  on  their  first  idea,  that  they  allow 
time  for  action  once  a  decision  Is  made,  and  so  on.  Let  them  know 
that  self-management  is  crucial  when  under  stress.  A  specific 
technique  to  foster  this  is  having  them  articulate  a  detailed  self- 
crltlque  (focused  specifically  on  their  decision-making)  after  their 
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practice  sessions.  Specificity  and  searching  for  candidate  cause-effect 
relations  between  their  decision  processes  and  the  observed 
outcomes  are  key. 

indicators 

Training  In  heuristic  decision  procedures  Is  appropriate  when  the 
trainees  will  have  to  make  decisions  under  stress  or  In  a  significantly 
limited  amount  of  time. 

Counterindicators 

If  out  In  the  real  world,  the  training  graduate  will  not  be  under  nign 
levels  of  stress,  it  makes  more  sense  to  train  them  in  use  of  more 
complex  and  accurate  decision  strategies. 

Common  problems 

The  point  Is  to  find  the  right  balance  between  being  certain  the 
trainees  can  perform  the  task  smoothly  under  stress  if  necessary,  yet 
sacrificing  accuracy  and  cool-headedness  as  little  as  possible.  If  you 
find  trainees  are  not  adequately  accurate,  Increasing  the  structure  of 
the  decision  process  may  be  necessary,  although  joining  this 
approach  with  others  (as  listed  below)  may  be  useful  as  well. 

Related  techniques 

Autorhatlclty 
Simulation 
Behavioral  modeling 
*  Stress 

Stress  -  Full  Information 
Stress  -  Graduated  Training 
Stress  -  Phased  Training 
Stress  -  Emotional  Training 
;  Stress  -  How  To  Practice 
Overlearning 

References 
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ENACTIVE  MASTERY 

Description 

Sometimes  learning  something  new  is  intimidating.  It  can  help  to  try 
to  limit  the  amount  of  Intimidation  to  improve  the  effectiveness  of 
training.  Enactive  mastery  is  a  way  to  increase  self-efficacy. 

How  to  do  it 

The  aim  Is  to  ensure  that  fears  are  not  realized  while  a  task  Is 
accomplished.  Even  when  a  task  is  successfully  completed,  self- 
efficacy  Is  lost  If  the  learner  is  intimidated  in  the  process.  Enactive 
mastery  is  achieved  by  creating  an  atmosphere  where  trainees 
become  comfortable  while  they  perform  training  tasks. 

Arrange  it  so  that  the  learner  knows  in  advance  what  they  will  be 
taught.  Set  goals  that  are  attainable  but  hard  so  that  progress  is 
measurable  and  felt  to  be  worthwhile.  Let  trainees  do  well  for  a 
while.  Introduce  challenges  when  they  feel  good.  Walt  until  they  can 
predict  and  manage  threats  or  problems.  Make  sure  they  actively 
participate  with  the  subject  matter  and  experience  success. 

It  makes  sense  to  use  this  technique  in  conjunction  with  approaches 
such  as  contract  theory  and  locus  of  control. 

Indicators 

This  Is  a  good,  general  approach  for  all  types  of  training.  It  Is 
especially  useful  for  beginners  encountering  new  fields. 

Counterindicators 

Enactive  mastery  may  not  be  necessary  for  short,  easy  task  training. 

Common  problems  ... 

Goals  may  not  have  been  clearly  understood  and  may  need  to  be 
restated.  It  may  be  Initlmldating  in  Itself  for  some  trainees  to  admit  to 
feeling  Initlmldated.  Techniques  such  as  establishing  an  environment, 
and  the  sections  on  positive  environment  may  prove  helpful  In 
overcoming  this. 

Relatedtechn  Iques 

Attitude:  Locus  Of  Control 
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Contract  Theory 
Establishing  An  Environment 
Socialization 

References 
Latham 
Gist  J.9B6 
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ESTABLISH  AN  ENVIRONMENT  FOR  TRAINING 

Description 

A  trainer  can  make  his  or  her  Job  a  lot  easier  by  establishes  clear 
boundaries  for  when  training  is  and  Is  not  happening.  Training  should 
be  a  definite  and  purposeful  activity.  In  this  way  It  becomes  a  habit 
that  Is  taken  seriously.  A  positive  set-up  Is  created  which  In  turn 
leads  to  a  positive  mood  which  has  been  shown  to  be  conducive  to 
learning.  Such  a  set  up  is  aided  by  establishing  a  training 
environment. 

How  to  do  it 

Make  sure  the  physical  environment  signals  the  need  to  concentrate 
and  work  hard.  The  environment  itself  cannot  produce  good  training 
habits,  but  It  should  not  Interfere  with  them,  Always  start  training 
with  the  same  simple  procedure  such  as  a  group  meeting,  or  a  lap 
around  the  gym.  Employ  an  end  routine  in  the  same  way. 

Breaks  are  thought  to  be  important  not  just  for  refreshment,  but  also 
as  a  tool  to  end  a  training  period  which  has  become  unproductive. 
Taking  o  break  can  help  to  avoid  the  association  of  training  with 
unproductive  activity  and  to  help  preserve  the  right  atmosphere. 

Make  sure  any  breaks  are  clearly  signaled  and  of  a  set  duration. 
There  1$  a  view  that  breaks  should  occur  within  the  same  physical 
boundaries  as  training.  This  ensures  continuity  and  fewer  lapses  from 
a  schedule.  As  a  training  routine  becomes  more  established,  trainees 
often  find  they  can  accomplish  more  and  need  fewer  breaks. 

Using  this  technique  produces  an  environment  which  reinforces  the 
habits  learned  in  training,  other  techniques  should  be  used  to  make 
sure  these  are  good  habits.  You  might  also  want  to  encourage  those 
being  trained  to  create  their  own  training  environment  when  they 
practice  without  you. 

Indicators 

This  approach  to  training  Is  appropriate  when  training  Is  long  term, 
and/or  Intensive.  It  Is  also  important  whenever  the  trainer  and 
trainee  are  new  to  each  other.  The  training  environment  matters 
whether  It  is  in  a  social  context,  a  work  setting,  with  a  team,  hands 
on  training,  or  any  other  type.  It  can  be  used  for  Instruction  or 
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practice. 

Counterlndlcators 

When  training  Is  to  be  light  and  short  or  whenever  a  high  level  of 
concentration  Is  not  needed. 

Common  problems 

People  can  develop  bad  training  habits,  A  change  to  the  training 
environment  may  be  needed  to  prevent  it  continuing  to  reinforce 

these. 

Related  techniques 

Mood 

Making  Training  Seem  Important 

Lead^By-Example 

Socialization 

Shaping 

Attitude,  Expectations 

References 

Domjan  &  Burkhard 
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FEEDBACK— WITHHOLD  IT 


As  a  general  rule,  it  is  good  to  give  trainees  feedback.  However,  it  is 
sometimes  beneficial  to  withhold  feedback/so  that  the  trainee  can 
learn  to  use  Intrinsic  cues,  either  from  within  himself  or  herself  or  from 
the  environment. 

How  to  do  It 

In  some  tasks,  extrinsic  feedback  can  be  weak,  ambiguous,  or  even 
misleading,  and  practitioners  must  be  sensitive  to  intrinsic  feedback 
to  adjust  and  adapt  their  actions.  When  this  Is  the  case.  It  is  useful  to 
withhold  feedback,  forcing  the  trainees  to  rely  on  Intrinsic  cues  about 
what  they're  doing  and  the  consequences  of  what  they  have  done, 

To  do  this,  feedback  should  be  withheld  on  some,  but  not  all,  of  the 
trials  during  a  training  session.  A  moderate  level  of  competence 
should  be  present  before  feedback  is  withheld,  so  that  trainees  are 
ready  to  develop  their  reliance  on  intrinsic  cues  without  too  much 
error  In  selection  or  Interpretation. 

"Too  much"  Is  admittedly  vague.  The  trainees  should  be  allowed  to 
make  mistakes  and  learn  from  them  ("if  knowledge  Is  power,  then 
error  Is  fuel").  On  the  other  hand,  It  Is  not  desirable  to  have  the 
trainees  learn  wrong  cues  or  wrong  cue  interpretations.  The  line 
must  be  found  between  enough  feedback  for  guidance,  but  not  so 
much  as  to  stunt  the  trainees'  learning  to  find  and  use  the  cues  for 
themselves. 

Since  withholding  feedback  Is  done  to  push  the  trainee  toward 
relying  on  Intrinsic  cues,  the  feedback  withheld  will  mostly  be  action 
feedback  (feedback  concurrent  with  the  person's  actions).  There  may 
be  cases  when  learning  feedback  (feedback  that  comes  at  task 
completion)  should  also  be  withheld.  If  the  structure  of  the  task 
Includes  repeated  episodes,  with  the  performance  on  each  sensitive 
to  the  performance  on  the  last,  withholding  learning  feedback  can 
encourage  development  of  good  judgement  regarding  quality  of  task 
(episode)  performance. 

Depending  on  the  task  and  the  trainees,  it  may  be  appropriate  to  tell 
them  you  are  going  to  withhold  feedback,  this  serves  two  purposes, 
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It  prevents  them  feeling  like  they  are  suddenly  or  inexplicably  being 
neglected.  And,  it  lets  them  know  they  have  a  learning  goal  of 
increasing  their  sensitivity  to  intrinsic  cues. 

The  feedback  at  issue  is  only  the  feedback  from  the  trainer  to  the 
trainee.  It  is  hard  to  think  of  circumstances  where  it  would  be  a  good 
idea  to  artificially  delay  or  withhold  environmental  feedback  that 
occurs  naturally  during  task  performance. 

Indicators 

Tasks  for  which  the  practitioner  must  monitor  and  judge  his  or  her 
own  progress  and  performance  in  the  absence  of  reliable  external 
feedback— tasks  where  workers  are  often  far  removed  from 
supervisors  or  coworkers,  for  example— withholding  feedback  is 
indicated. 

Counterindicators 

If  the  nature  of  the  task  is  such  that  the  external  cues  are  sufficiently 
strong,  then  withholding  feedback  may  slow,  rather  than  speed,  the 
learning  process.  For  example,  if  in  task  performance  there  is 
immediate  and  obvious  feedback  from  the  environment  about 
whether  the  act  worked  or  not,  then  the  feedback  the  trainer  is  going 
to  give  is  likely  to  be  about  how  to  perform  the  action,  not  what 
action  to  perform  or  what  cues  to  attend  to.  When  this  is  the  case, 
withholding  feedback  may  confuse  the  trainee,  or  may  allow  bad 
habits  to  become  ingrained. 

Common  problems 

Be  very  careful  with  when  and  to  what  degree  feedback  is  withheld. 
Don't  withhold  feedback  that  would  be  difficult  for  the  trainee  to  find 
on  his  or  her  own.  When  particular  trainee  actions  if  left 
uncommented  would  likely  be  misinterpreted,  provide  the  needed 
feedback.  While  it's  better  if  they  learn  for  themselves,  the  trainer 
must  be  sensitive  to  the  rate  of  learning  and  the  potential  for 
mislearning. 

If  the  trainees  are  not  picking  up  the  intrinsic  cues  of  the  task  even 
with  the  trainer's  guidance  and  feedback,  then  they  are  not  ready  for 
feedback  to  be  withheld. 

Related  techniques 
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Feedback— Delay 
Advance  and  Back  Off 
^Things  That  Get  You  To  Rely  On  Cues 
Immediate  Practice 
Learn  By  Doing  Only 

References 
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FEEDBACK— PAUSES  &  DELAYS 

Description 

While  feedback  permeates  training,  there  are  times  when  it  is  good 
to  pause  after  a  training  episode  before  giving  feedback,  or  simply  .to 
delay  giving  the  feedback  during  the  training  performance. 

Pauses  and  delays  are  useful  to  the  extent  that  they  make  it  easier 
for  the  trainee  to  absorb  and  assimilate  the  feedback.  Sometimes  it 
takes  time  to  understand  and  integrate  feedback.  Pausing  provides 
this  kind  of  breathing  room.  And  sometimes  it  is  better  for  the  trainee 
to  form  his  or  her  own  opinion  about  what  has  happened  before 
hearing  the  trainer's  view.  Delaying  supports  this. 

How  to  do  it 

Pauses  and  delays  are  cousins,  and  both  useful  to  allow  trainees  to 
process  feedback  information,  whether  to  guide  current  performance 
or  to  plan  their  next  action.  Pauses  are  applied  to  learning  feedback, 
and  delays  to  action  feedback  (see  Feedback). 

A  pause,  correctly  applied,  comes  after  post-training  episode 
feedback.  The  trainees  perform  the  task,  receive  feedback,  an 
appropriately-timed  pause  is  given,  and  they  are  sent  on  to  next 
practice  trial.  The  pause  should  not  be  between  the  end  of  the 
training  episode  and  the  receipt  of  feedback.  Pausing  then  doesn't 
really  help  the  trainees,  and  it  can  be  interpreted  as  anger  or 
punishment. 

If  the  task  is  complex,  going  over  trainees'  performance  before 
providing  feedback  can  clarify  the  relationship  between  actions  and 
observed  results.  In  addition,  it  provides  an  opportunity  to  highlight 
the  most  important  factors.  This  will  make  understanding  and 
applying  the  feedback  more  straightforward  for  the  trainees.  It  may 
also  break  up  any  tendency  trainees  may  have  to  "lock  on"  to  a 
particular  interpretation  of  their  actions,  to  the  possible  exclusion  of 
the  trainer's  presumably  more  clear-eyed  appraisal. 

If  the  task  is  such  that  delayed  feedback  doesn't  become  instantly 
stale,  either  because  the  action  of  interest  has  been  superseded  by 
a  dozen  others,  or  because  the  stress  level  in  the  task  makes  the 
mental  effort  of  associating  delayed  feedback  with  the  right  action 
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too  great,  then  allowing  a  delay  can  let  the  trainee  practice  self¬ 
generating  feedback.  This  will  almost  always  be  a  desirable  training 
goal.  The  length  of  the  wait  will  depend  on  the  trainer's  judgement 
about  both  the  task  and  the  trainee,  but  the  point  is  to  wait  just  long 
enough  that  the  trainee  has  formed  his  or  her  own  opinion  of  what  is 
going  on.  Then  when  they  receive  the  feedback  they  have  something 
vivid,  something  of  their  own,  to  compare  it  with.  Not  only  does  this 
serve  immediate  task  improvement,  but  the  trainer's  feedback  now 
serves  the  dual  purpose  of  illuminating  both  the  task  and  the 
trainee's  growing  skill  at  self-assessment  and  self-correction. 

All  of  the  feedback  discussed  above  is  that  from  trainer  to  trainee. 
Artificially  delaying  environmental  feedback  that  would  otherwise 
occur  naturally  during  task  performance  is  not  likely  to  be  helpful, 
unless  during  the  early  stages  of  training  the  trainees  need  time  to 
prepare  to  receive  and  use  the  feedback. 

Pauses  and  delays  can  help  trainees  use  feedback  smartly.  "Smartly" 
doesn't  necessarily  mean  consciously,  since  substantial  learning 
takes  place  without  conscious  awareness.  But  having  an  extra 
moment  to  process  self-generated  feedback  during  task  performance, 
or  to  let  the  trainer's  feedback  sink  in,  can  accelerate  learning. 

Indicators 

Pausing  or  delaying  feedback  is  useful  when  the  trainee  is  likely  to  be 
able  to  use  the  pause  to  formulate  and  test  his  or  her  own 
hypotheses  about  actions  and  results.  It  may  also  be  useful  in  tasks 
where  the  performer  has  a  chance  to  correct  or  compensate  for 
errors  made  earlier  in  task  performance. 

If  a  substantial  amount  of  internal  control  and  self-optimization  is 
required  for  the  task  (as  is  often  the  case  in  athletics),  pauses  and 
delays  can  allow  the  trainees  to  experiment  more  freely  to  find  their 
own  balance.  The  risk  is  in  letting  potential  bad  habits  take  root  and 
grow. 

Counterindicators 

Don't  get  carried  away.  Usually,  feedback  should  either  be  immediate 
(action  feedback)  or  in  close  proximity  to  completion  of  a  training 
session  (learning  feedback).  If  because  of  the  nature  of  the  task  it  is 
difficult  for  trainees  to  see  the  relationships  between  their  actions 
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and  the  results,  delaying  or  pausing  feedback  is  probably  not  going 
to  be  a  good  idea. 

Common  problems 

Trainees  accustomed  to  immediate  feedback  can  interpret  it's  delay 
as  disapproval.  Warning  the  trainees  beforehand  of  the  intention  to 
delay  can  ameliorate  this. 

Allowing  the  trainees  to  begin  another  task  before  providing 
feedback  on  the  last  and  allowing  them  to  digest  it  can  dilute  the 
effect,  even  to  the  point  of  making  the  feedback  useless,  or  worse. 

Related  techniques 

Feedback— Withhold 
Also  see: 

Reinforcement 
Rewards  &  Punishment 

References 

Holding 


Syukhtun  Research,  Inc. 


214 


4 


GAMES  AND  SIMULATIONS 

Description 

Games  and  simulations  are  ways  of  letting  people  practice  the  task 
they  are  training  for  without  actually  doing  it.  The  information  below 
is  about  how  to  use  games  and  simulations,  but  not  how  to  design 
(or  in  some  cases,  afford)  them. 

How  to  do  it 

There  are  many  types  of  simulations.  The  point  of  a  simulation  is  to 
provide  an  acceptably  realistic  task  environment  when  the  actual 
environment  is  not  available  or  too  dangerous  for  trainees  to  practice 
in.  The  key  thing  is  that  the  actions  taken  by  the  trainees  do  not 
have  the  effect  on  the  world  that  they  would  in  real  life. 

A  simulation  is  elaborate  practice.  It  requires  preparation,  planning, 
and  debriefing,  especially  when  it  is  complex.  One  view  holds  that  it  is 
best  to  start  out  with  lectures  and  examples.  These  can  be  followed 
with  demonstrations  and  games  or  simple  simulations.  It  very  much 
depends  on  the  content  of  the  task.  For  some  tasks  a  very  great  deal 
of  preparation  is  necessary  before  a  simulation  can  be  productively 
(or  safely)  used. 

A  game  can  be  a  simulation  or  it  can  be  an  activity  where  the  skills 
needed  are  used,  but  not  in  the  task  itself  or  even  in  a  situation  that 
simulates  the  actual  task. 

Selection  of  the  game  or  simulation  is  important.  Ensure  it  contains 
enough  feedback,  pointers  and  instruction  to  make  your  assistance 
minimal.  Let  the  simulation  or  game  run.  Be  on  hand  to  answer 
questions  or  to  handle  emergencies  depending  on  the  nature  of  the 
training.  Afterwards,  apply  feedback,  reinforcement,  coaching,  and 
practice  with  regard  to  specific  task  components.  Just  as  with 
practices,  do  not  rely  on  the  simulation  to  do  it  all  for  you.  The  results 
or  built-in  tests  may  themselves  show  whether  or  not  tasks  have 
been  learned.  If  not,  supplementary  tests  can  be  provided  to  ensure 
relevant  knowledge  has  been  gained. 

Indicators 

If  you  have  a  dangerous  task  that  you  want  people  to  be  able  to 
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practice  without  danger,  you  would  be  well  served  by  using  a 
simulation.  A  simulation  is  also  useful  if  the  task  is  costly  or  does  not 
naturally  occur  often  enough  to  provide  consistent  practice. 

Games  may  be  helpful  in  making  dull  tasks  more  fun.  They  can  also 
encourage  people  to  perform  well.  Simulations  are  helpful  precursors 
to  on-the-job  training. 

Counterindicators 

Simulations  can  be  expensive.  They  can  also  be  used  beyond  the 
point  where  on-the-job  training  should  be  started. 

Common  problems 

Using  a  simulation  as  a  beginning  training  tool  will  make  training  too 
hard.  The  learner  may  miss  the  underlying  causes,  meanings  and 
relationships.  Also,  trying  to  learn  while  being  highly  active  can  cause 
problems  because  no  time  is  allowed  for  reflection.  Remember  that 
simulations  in  particular  are  designed  for  the  application  and  support 
of  learned  information,  not  for  basic  learning  itself. 

The  simulation  should  get  the  important  cues  right  as  far  as  safely 
possible.  It  should  not  be  possible  to  learn  to  be  effective  at  the 
simulation  without  corresponding  growth  of  skill  at  the  real  task.  Said 
another  way,  it  shouldn't  be  possible  to  "beat"  the  simulation  using 
any  skills  specific  to  the  simulation. 

Related  techniques 

Computer  Aided  Instruction 
Deliberate  Practice 
Stress  -  How  To  Practice 

References 
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GROUPS:  COOPERATIVE  LEARNING 

Description 

This  technique  applies  to  learning  as  a  cooperative  process.  This  is 
encouraged  by  giving  the  whole  group  an  equal  reward  based  on  its 
performance  or  improvement  as  whole.  A  reward  structure  for  groups 
is  used  whether  or  not  members  will  be  doing  the  task  together. 

How  to  do  it 

Divide  trainees  into  groups.  This  can  be  done  on  a  basis  of  similar 
ability  levels  if  task  performance  alone  is  a  priority,  but  greater 
cooperation  will  be  learned  if  group  members  are  of  mixed  ability. 
Socialization  and  buddy  techniques  can  help  in  group  selection. 
Competition  is  avoided  if  groups  are  assigned  different  tasks. 

Ensure  each  group  is  clear  about  its  task  expectations  and  any  time 
constraints.  Explain  the  reward  criteria.  These  should  be  based  on 
task  component  skills  and  completion,  improvement  on  any  previous 
efforts,  and  overall  performance.  In  addition,  good  training  skills  and 
attributes  should  also  be  rewarded  such  as  use  of  feedback  and 
positive  attitudes. 

Monitor  the  groups  while  they  work  on  the  task  and  as  they  use  their 
learning  and  training  skills.  Be  on  hand  to  answer  questions.  As  in 
paired  post-training,  it  may  help  to  end  with  feedback  from  all  the 
groups  about  their  experiences.  Give  rewards  to  each  group 
individually,  but  in  a  consistent  fashion. 

Research  shows  that  cooperative  group  training  can  be  more 
effective  for  most  tasks  than  competition  or  individualization. 
Rewarding  in  a  group  structure  aims  to  help  people  learn  from  each 
other.  It  also  helps  them  learn  to  purposefully  train  each  other.  There 
is  a  view,  too,  that  group  rewards  are  especially  helpful  to  slower 
trainees. 

With  any  group  rewards,  some  observers  believe  it  is  still  important 
to  retain  some  individual  accountability.  This  is  thought  to  prevent 
those  of  higher  ability  not  trying  and  those  of  lesser  ability  from  ' 
failing  to  contribute.  Individual  accountability  could  occur  more  in 
terms  of  performance  reviews  and  advancement  than  explicit  reward 


Syukhtun  Research,  Inc. 


217 


structures. 


It  is  best  to  strike  a  balance  between  reward  structures  for  groups 
and  individual  training.  This  ensures  trainees  receive  individual 
attention  and  customized  training. 

Indicators 

This  approach  is  considered  useful  for  most  group  training  situations. 
It  is  especially  useful  when  a  high  degree  of  cooperation  is  needed 
for  task  performance,  or  when  trainees  are  learning  how  to  train. 
Similarly,  reward  structures  for  groups  can  be  used  if  competition 
would  hinder  task  learning,  for  example  leaving  slower  trainees 
behind,  or  encouraging  an  attitude  that  success  depends  on  others 
failing.  Rewarding  groups  can  also  aid  socialization  if  you  are 
introducing  new  members  to  the  larger  group. 

Counterindicators 

Very  large  groups  and  numbers  of  groups  are  hard  to  monitor.  Some 
tasks  to  be  completed  alone  are  learned  more  efficiently  alone.  A  high 
degree  of  preparation  and  equipment  is  sometimes  needed.  Group 
rewards  are  not  useful  when  a  task  requires  customized  training. 

Common  problems 

Group  training  can  fail  if  group  norms  dictate  that  some  or  all  of  the 
trainees  not  do  their  best.  Insufficient  monitoring  and  reward  can 
also  encourage  goof-off  and/or  demoralization. 

Related  techniques 

Teams  -  General  methods 
Teams  -  Cooperative  controversy 
Paired  post-training 

References 
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INTERACTIVE  IMAGERY 

Description 

Remembering  images  instead  of  words  can  make  recall  easier.  When 
you  have  to  remember  a  list  of  items,  it  is  a  good  idea  to  have  them 
trigger  each  other.  That  way,  you  only  have  half  (or  less)  the  amount 
to  remember. 

How  to  do  it 

If  you  are  trying  to  memorize  a  list  of  nouns,  do  it  in  pairs  and  create 
mental  images  showing  the  items  together  in  an  interactive  way. 
They  will  trigger  each  other. 

So  maybe  you  have  to  buy  a  rabbit,  a  xylophone  and  some  matches 
at  the  store.  To  remember  this  list,  picture  a  rabbit  playing  the 
xylophone  with  some  matches  (on  the  clerks  head,  if  you  want). 

This  will  make  it  easier  to  remember  all  of  the  items  because  you  will 
only  have  to  remember  one  thing.  Also,  there  will  be  an  interesting 
image  to  remember,  so  it  will  be  easier  to  remember  (maybe). 

This  works  most  easily  with  nouns  -  things  of  which  you  can  form 
images. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 

References 
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LEARN  BY  DOING  ONLY 

Description 

Do  not  instruct.  Simply  allow  the  person  to  progress,  and  give  them 
challenges  until  they  reach  a  sufficient  level  that  training  can  be 
terminated. 

How  to  do  it 

Tell  the  trainee  to  start  doing  it. 

It  helps  if  they  set  goals,  especially  if  it  is  an  individual  activity.  Goals 
provide  feedback  on  progress  to  the  person,  as  well  as  opportunites 
for  improvement.  It  is  important  to  keep  motivation  high,  by  having 
fun  doing  it  and  seeing  progress. 

Learners  should  be  able  to: 

1.  Model  their  behavior  on  others  who  are  already  competent  at  the 
task. 

2.  Participate  or  compete  with  those  with  more  experience,  so  that 
mistakes  are  made  visible  and/or  admonished. 

To  be  effective,  there  must  be  substantial  feedback  intrinsic  to  the 
task  and  the  environment  must  be  stable  and  appropriate. 

Indicators 

This  technique  is  appropriate  when  through  minimally  supervised 
practice  trainees  can  achieve  an  adequate  level  of  competence. 

Counterindicators 

The  easier  it  is  to  get  feedback  from  the  environment,  the  better  this 
technique  will  work.  The  technique  is  inappropriate  if  bad  habits  are 
easy  to  develop. 

Common  problems 

If  it  is  easy  to  get  frustrated,  or  there  is  significant  risk  of  injury,  this 
is  inadvisable. 

Related  techniques 

Limit  Introduction 
Learn  By  Mucking  About 
Feedback  -  Delay  It 
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Feedback  -  Withhold  It 
Path  Goal  Theory 

References 

Ericsson 


r 


Syukhtun  Research,  Inc. 


221 


LET  TRAINEE  BECOME  MASTER 

Description 

Initially  trainees  do  best  with  a  lot  of  external  support  in  the  form  of 
motivation,  guidance,  teaching  and  feedback.  Good  training  develops 
self-motivation,  reliance,  direction,  and  practice  ability.  Eventually 
students  are  ready  to  direct  their  own  training,  to  become  their  own 
masters.  Support  or  competition  is  then  requested  by  trainees  only 
as  it  is  felt  to  be  needed. 

How  to  do  it 

Gradually  decrease  environmental  support,  and  start  to  implement 
training  that  stresses  good  practice  skills  (see  guided  practice  alone), 
self  monitoring,  evaluation  and  feedback.  Equally  important  are  self 
regulation  and  organization,  and  the  ability  to  identify  progress  and 
errors. 

The  goal  is  to  reach  a  point  where  learning  is  designed  and  controled 
by  the  trainee.  Once  trainees  are  highly  motivated,  familiar  with  and 
very  comfortable  using  training  techniques  independently,  they  are 
ready  to  be  their  own  masters.  Learners  will  then  take  advantage  of 
competition,  coaching  and  social  interaction  as  needed.  Training 
requested  is  likely  to  be  improved  because  salience  of  feedback 
should  increase  along  with  productivity  and  amount  of  practice  if 
trainees  were  well  prepared.  Increased  strength  of  motivation, 
confidence,  and  self  efficacy  may  also  follow. 

Work  on  building  a  strong  basis  in  fundamentals  since  this  will  be 
essential  for  the  student  to  be  able  to  self  correct.  Guided  practice 
alone  can  be  followed  with  short  sessions  or  simple  tasks  where  the 
student  is  master.  This  may  help  to  build  confidence,  ensure  that 
mistakes  are  not  practiced,  and  allow  time  for  post-session  feedback. 
Be  prepared  to  step  in  on  request,  if  mistakes  result,  or  if  learning 
slows  down  considerably.  More  practice  and  observer  feedback  may 
be  needed. 

Trainee  as  master  is  based  on  research  which  shows  that  people 
learn  learn  best  if  they  have  opportunities  for  problem  solving, 
analogy  making,  extended  inferencing,  interpretaton,  and  for  working 
in  unfamiliar  situations  requiring  transfer.  Deep  understanding  of  how 
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things  work  is  best  achieved  when  people  are  totally  involved  and 
solve  problems  themselves.  This  does  not  occur  if  a  trainer's 
presence  distracts  from  total  involvement  or  if  a  trainer  always 
provides  answers. 

Indicators 

Student  as  master  becomes  applicable  during  long  term  training 
which  calls  for  large  improvements  in  ability.  It  is  not  necessary  that 
the  trainer  and  trainee  have  spent  a  long  time  together.  Both  high 
trainee  motivation  and  good  mastery  of  basics  signal  the  time  for 
practice  alone  followed  by  student  as  master. 

Counterindicators 

Some  tasks  require  a  trainer  or  opponent  to  provide  expert  and/or 
audience  feedback.  This  is  true  especially  of  performance  activities 
such  as  dance.  Trainees  of  a  young  age  may  also  require  external 
support  even  after  they  reach  an  expert  level. 

Common  problems 

The  trainer  has  no  right  to  observe  once  supervised  training  has 
been  completed,  unless  invited  to  do  so.  However,  potential 
problems  are  lack  of  motivation,  practicing  mistakes,  and  inefficient 
practice.  If  you  notice  that  trainees  deviate  from  good  training  habits, 
you  may  want  to  work  on  the  areas  involved. 

Related  techniques 

Deliberate  practice 
Practice  alone 

References 
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LIMIT  INTRODUCTORY  INSTRUCTION- 
PRACTICE  INSTEAD 

Description 

This  is  a  way  of  training  someone  to  do  a  procedural  skill.  When 
learning  a  procedure,  it  is  good  to  have  someone  explain  it  to  you, 
but  it  is  necessary  to  practice  actually  doing  it  yourself.  The 
suggestion  here  is  that  you  limit  the  introduction  students  get 
through  lecture,  books,  etc.,  instead  giving  them  something  brief,  and 
then  let  them  actually  do  the  thing. 

How  to  do  it 

Limit  the  amount  of  introductory  instruction,  and  let  them  do  it.  Often 
times  you  use  instruction  (as  opposed  to  practice)  as  an  introduction 
to  the  material.  The  suggestion  is  they  will  learn  it  as  declarative 
knowledge  if  you  do  this,  whereas  if  the  introduction  is  brief,  and 
then  you  let  them  begin  practice,  they  will  learn  it  as  a  procedure, 
which  is  good  if  what  you  want  them  to  remember  is  a  procedure. 

Give  them  feedback  during  the  practice  so  that  they  get  it  right. 
Immediately  correct  errors.  You  are  allowing  them  to  do  things  any 
way  they  want,  so  you  have  to  be  careful  that  they  do  them  right. 

But,  if  you  start  out  by  giving  a  lot  of  negative  feedback,  then  they 
are  going  to  learn  that  the  task  it  hard,  they  are  not  good  at  it,  and 
that  it  is  not  enjoyable.  So,  don't  point  out  every  mistake  at  first,  and 
emphasize  positive  feedback. 

Indicators 

Any  time  you  are  trying  to  teach  a  procedure. 

Counterindicators 

This  is  not  suitable  for  a  complex  cognitive  tasks,  or  tasks  where 
there  is  very  little  feedback  intrinsic  to  the  task. 

Common  problems 

By  overly  limiting  introduction  the  student  may  learn  improper  form  or 
habits. 

Related  techniques 
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Learn  By  Mucking  About 
Learn  By  Doing  Only 
Immediate  Practice 
Adaptive  Training 
Stress  -  Full  Information 
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MAKING  TRAINING  SEEM  IMPORTANT 
(MANAGING  TRAINING  IN  ORGANIZATIONS) 


Description 

Sometimes  trainees  are  motivated,  sometimes  they  don't  feel  like 
trying  that  hard,  and  sometimes  they  think  you  have  nothing  to  offer 
and  are  resentful  at  just  being  in  training  at  all.  This  section  suggests 
different  ways  to  convince  trainees  that  the  training  they  are  going  to 
receive  is  useful  and  worth  effort,  and  that  they  should  try  to  make 
use  of  it  on  the  job  and  learn  something. 

How  to  do  it 

There  are  a  few  things  you  can  do. 

1.  Be  a  supportive  supervisor 

Cohen  (1990)  found  that  if  you  have  supportive  supervisors,  you 
enter  training  believing  it  will  work.  They  can  show  support  by 
discussing  the  training  with  the  employee,  establishing  goals, 
providing  release  time  to  prepare,  and  encouraging  the  employee. 
Also  from  this  study,  it  seems  that  those  who  set  goals  entered  with 
higher  motivation  to  learn,  but  based  on  the  nature  of  the  study,  it  is 
impossible  to  determine  causality. 

2.  Follow-up  /  assessment 

Baldwin  and  Magjuka  (1991)  found  that  if  trainees  enter  training 
expecting  a  follow  up  activity  or  assessment,  they  will  report  stronger 
intentions  to  transfer  what  they  were  going  to  learn  back  to  their 
jobs. 

3.  Long  term  importance  of  training 

*  A  recurrent  theme  is  that  how  you  treat  one  training  program  will 
effect  other  ones.  If  employees  get  the  impression  that  the 
managers  and  trainers  care,  then  they  will  try  to  learn.  If  they  have 
the  impression  that  'training'  means  'goof  off  and  learn  nothing'  then 
they  won't.  Try  to  set  the  right  atmosphere  in  the  first  place. 

Indicators 

This  section  is  intended  for  managers  who  have  employees  to  be 
trained. 
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If  you  get  the  feeling  that  your  people  don't  have  any  respect  for 
training,  or  if  you  think  they  could  be  getting  more  out  of  training, 
then  you  want  to  use  the  techniques  above. 

Counterindicators 

If  there  the  problem  is  out  of  your  control,  there  may  be  little  you  can 
do. 

Common  problems 

It  can  be  easy  for  trainers  to  act  like  bosses  at  times.  This  may  be 
hard  to  avoid,  especially  in  situations  where  trainees  are  likely  to 
treat  you  like  their  boss.  Use  socialization  and  positive  environment 
techniques  to  lessen  these  effects. 

Related  techniques 

Goals  -  Learning  Vs  Performance 

Socialization 

Training  Different  Ages 

Contract  Theory 

Choices  About  Training 

Lead  By  Example  -  Attitude 

Establish  Environment 

References 
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MATCH  THE  COMPLEXITY  OF  INSTRUCTION  AND 
PRACTICE 

Description 

Design  training  so  that  trainees  practice  what  you  teach  them,  by 
making  sure  that  the  practice  is  appropriate  for  the  instruction  you 
have  given  and  vice  versa.  If  the  instruction  is  complete,  let  them 
practice  the  complete  task.  If  the  instruction  is  simple,  make  the 
practice  simple.  What  they  learn  in  instruction,  they  should  practice. 

How  to  do  it 

The  important  thing  is  to  make  sure  that  practice  is  designed  so  that 
they  will  use  what  you  teach  them.  If  the  instruction  and  practice  are 
not  matched,  the  trainees  will  not  be  able  to  make  use  of  what  they 
learn  during  instruction,  and  therefore  it  will  not  be  learned  as  well. 

Indicators 

The  greater  the  distance  between  instruction  and  practice,  measured 
in  either  time  or  location,  the  more  important  this  is. 

Counterindicators 

If  instruction  and  practice  are  concurrent,  this  tends  to  solve  itself.  If 
there  is  a  practice  that  also  requires  a  distinct  explanatory 
component,  this  may  not  be  relevant. 

Common  problems 

If  instruction  and  practice  are  mismatched,  lower  efficacy  will  result. 

Related  techniques 

Immediate  practice 
Don't  take  your  time 
Operationalize  Definitional  Concepts 

References 
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MENTAL  SIMULATION 

Description 

When  there  is  restricted  opportunity  to  practice  something,  an 
alternate  technique  is  to  simulate  practice  mentally.  If  you  do  this 
realistically,  research  shows  that  it  produces  measureable  learning  in 
many  skills. 

How  to  do  it 

The  practicer  imagines  the  situation  as  accurately  as  possible,  trying 
to  include  all  of  the  important  details.  Then,  he  or  she  imagines  as 
vividly  as  possible  the  feeling  of  doing  the  task  exactly  right.  This  is 
repeated,  as  would  be  done  in  a  real  practice  session. 

Indicators 

Mental  simulation  improvements  are  usually  made  in  "motor 
programming,"  that  is,  in  complex  physical  action. 

Counterindicators 

Mental  simulation  requires  intense  concentration,  often  even  more 
than  is  necessary  in  actual  task  performance.  As  a  result,  it  is  tiring, 
and  not  everyone  can  do  it  effectively. 

Common  problems 

Inattention  and  inaccuracy  are  the  most  common  problems. 

Related  techniques 

Deliberate  practice 

References 
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MENTORS 


Description 

A  mentor  is  someone  who  has  experience  in  the  task  or  the 
organization  the  trainee  is  entering.  Having  a  mentor  can  be  very 
helpful  for  people,  and  this  section  describes  how  mentor 
relationships  can  be  created  by  trainers  or  managers,  how  mentors 
are  most  effective,  and  how  to  train  mentors. 

There  is  a  large  range  in  what  can  be  called  a  mentor.  A  mentor  can 
just  be  a  high  ranking  friend  who  gives  you  a  little  support  when  you 
need  it,  or  it  could  be  someone  who  is  intimately  involved  in  your 
career  choices.  Also,  there  is  a  lot  of  range  in  "assigning  mentors." 
This  could  be  anything  from  posting  assignments  on  a  bulletin  board 
for  every  member  of  the  organizations  to  a  friendly  word  of 
encouragement  about  someone's  mentor  relationship. 

How  to  do  it 

The  best  mentorships  are  usually  the  ones  that  develop  on  their 
own,  not  the  assigned  ones.  Tannenbaum  and  Yukl  say  that  the  best 
thing  is  when  mentor  relationships  occur  naturally,  but  this  doesn't 
always  happen,  and  sometimes  they  are  assigned  instead.  When 
this  happens,  though,  there  is  the  possibility  of  conflict  and  lack  of 
commitment  by  the  mentor  -  unless  assignments  and  mentor  training 
are  done  carefully.  If  you  assign  a  mentor  from  whom  the  junior 
person  learns  to  avoid  hard  work  and  to  hate  it,  you  are  going  to  get 
a  poorly  socialized  protege. 

Zey  (1988)  recommends  more  use  of  mentors  for  difficult 
adjustments,  such  as  to  a  new  job,  assignment,  or  country.  Noe 
(1988a)  and  Kram  (1985)  found  the  same  thing,  which  is  that 
mentors  serve  two  functions  -  one  is  psychosocial  and  includes 
acceptance,  encouragement,  coaching,  counseling.  The  other  is 
career  oriented,  and  includes  sponsorship,  protection,  challenging 
assignments,  exposure  and  visibility.  Noe  found  that  people  with 
assigned  mentors  get  the  psychosocial  but  not  the  career  benefits. 

Tannenbaum  and  Yukl  say  that  instead  of  putting  effort  into  making 
these  assignments,  or  at  least  along  with  it  that  effort,  effort  should 
be  put  into  improving  the  effectiveness  of  the  mentors.  As  far  as  how 
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to  train  them,  they  identify  six  issues  without  actually  making 
recommendations  with  respect  to  how  to  train  them: 

•  How  to  identify  an  appropriate  protege. 

•  Phases  of  mentorship  and  when  to  let  go. 

•  How  to  coach  without  being  overbearing. 

•  Rewards  of  being  a  mentor. 

•  How  to  reinforce  company  goals  and  values. 

•  How  to  work  with  a  protege  who  is  different  from  you  (e.g.,  age, 
race,  and/or  gender). 

Female  managers  tend  to  encounter  stereotypes,  which  causes 
limited  contact  with  potential  mentors,  and  stereotypical  attitudes 
toward  the  women  by  potential  mentors,  tokenism,  gender 
differences  in  socialization  practices,  norms  about  cross-gender 
interactions/relationships,  and  differences  in  the  acquisition  and  use 
of  power.  Successful  mentoring  can  ameliorate  some  of  these 
problems. 

If  you  assign  everyone  a  mentor,  the  people  who  are  the  mentors 
are  going  to  be  doing  a  lot  of  mentoring.  The  may  thinking  of  the 
person  they  are  mentoring  as  special  and  just  think  of  them  as  one  in 
a  string  of  many  people  that  they  do  the  same  thing  for,  and 
therefore  they  probably  won't  do  a  very  inspired  job. 

Sometimes,  training  someone  to  be  a  mentor  might  be  a  bad  idea. 
Effective  mentoring  is  such  a  subtle  skill  that  some  people  may  be 
better  off  just  doing  it  the  way  they  know  how.  If  they  try  to  follow 
some  set  of  rules  or  whatever  you  teach  them,  they  may  just  become 
artificial. 

Related  techniques 

Socialization 

Lead  By  Example  -  Behavior 

Lead  By  Example  -  Attitude 

Behavioral  Modeling 

Advance  And  Back  Off 

Paired  Post-Training 

Don't  Take  Your  Time  -  Heuristics 
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METHOD  OF  ELABORATION 

Description 

This  is  a  way  of  making  basically  meaningless  information  memorable, 
by  assigning  meaning  to  it  arbitrarily. 

How  to  do  it 

Make  up  an  interesting  sentence  that  includes  what  you  want  to 
remember  (not  necessarily  in  an  informative  way).  For  example,  "the 
car  salesman  returned  the  first  car"  is  a  way  to  remember  that  the 
command  "car"  in  the  computer  programming  language  Lisp  returns 
to  the  first  item. 

You  can  make  up  pretty  complex  things  using  the  method  of 
elaboration.  The  problem  is  that  you  probably  want  to  remember 
these  things  for  a  while,  and  the  phrase  "the  car  salesman  returned 
the  first  car,"  while  more  interesting  than  the  plain  information  that 
the  car  command  returns  the  first  item,  is  still  pretty  boring.  If  you 
have  no  context  to  understand  it  in,  and  nothing  to  trigger  that 
sentence,  it  is  still  going  to  be  hard  to  remember  the  command.  You 
want  to  make  the  things  as  easy  as  possible  to  trigger. 

Indicators 

This  is  useful  when  you  have  meaningless,,  low  meaning,  or  arbitrary 
information  to  remember.  The  whole  point  is  to  assign  meaning  to 
make  things  memorable,  so  it  is  not  so  useful  if  there  is  already 
meaning  there  for  you  to  remember. 

Since  each  instance  of  this  technique  is  different,  it  can  be  used  for 
long  term  memorization. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 
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THE  METHOD  OF  LOCI 

Description 

This  is  a  technique  for  memorization.  If  you  have  a  list  of  things  to 
remember,  you  can  associate  each  one  with  a  different  point  on  a 
well-remembered  walk,  and  then  as  you  go  through  the  walk 
mentally,  the  things  you  want  to  remember  will  come  to  you. 

How  to  do  it 

The  first  step  is  to  memorize  the  walk. 

Choose  a  set  of  landmarks  that  you  always  use,  so  that  you  don't 
"walk  by  an  idea"  without  noticing.  Memorize  them  thoroughly. 

Then,  take  each  thing  that  you  want  to  remember,  and  try  to  form  a 
mental  association  between  it  and  the  next  landmark. 

The  most  effective  associations  may  be  images.  If  you  are  trying  to 
remember  bread,  imagine  a  loaf  of  bread  at  your  first  landmark.  If  the 
idea  is  more  complex,  you  may  be  able  to  come  up  with  an  image 
anyway  (use  lions  killing  slower  gazelles  for  natural  selection,  for 
example). 

When  you  are  ready  to  remember  them,  go  along  the  walk  mentally 
and  see  what  ideas  you  come  to. 

The  Pegword  Method  is  a  variation  on  the  method  of  loci.  Memorize  a 
set  list  of  objects  (e.g.,  bun,  Shoe,  tree)  to  be  your  "pegwords."  Use 
the  objects  that  you  have  memorized  just  like  you  use  the  landmarks 
on  the  walk:  Associate  the  things  to  remember  with  the  pegwords  by 
forming  mental  images.  Then,  when  you  are  at  the  store,  think  of  the 
pegwords,  and  you  will  remember  what  to  buy.  For  example  when 
you  think  of  bun,  you  will  know  that  you  have  to  buy  carrots  because 
you  have  an  image  of  a  hot  dog  bun  with  a  carrot  in  it. 

You  can  use  other  variations  if  you  want,  like  the  steps  of  getting 
ready  for  bed  (brush  teeth,  take  out  contacts,  turn  off  lights  etc.) 

Indicators 

This  is  not  as  much  for  memorizing  an  unordered  body  of  facts  as  for 
recalling  lists  of  items  or  ideas.  Despite  what  we  said  above,  the 
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ideas  don't  have  to  be  simple  -  the  Greeks  used  this  technique  to 
remember  the  points  they  wanted  to  make  in  speeches,  and  the 
points  were  undoubtedly  complicated. 

This  can  be  used  to  remember  ordered  lists  or  sequences  of  actions. 
You  could  associate  each  landmark  with  one  of  the  steps  of  making 
sure  an  airplane  is  ready  for  takeoff,  for  example. 

This  can  be  used  to  memorize  things  that  you  only  need  to  know 
once,  such  as  a  shopping  list,  and  that  is  what  it  is  intended  for. 

References 
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MOLD  MENTAL  MODELS  WITH  QUESTIONS 


Description 

This  method  aims  to  teach  complicated  information  through  careful 
selection  and  timing  of  questions.  Questions  requiring  less 
information  are  asked  at  first,  simplifying  things.  Questions  become 
progressively  harder,  in  such  a  way  that  the  learner's  mental  model 
grows  into  the  desired  form. 

How  to  do  it 

The  instructional  task  is  to  design  the  early  mental  models  so  that 
they  can  be  transformed  into  the  more  advanced  ones  with  minimum 
adjustment.  The  means  to  this  is  through  explanation  and  questions 
that  will  foster  mental  models  in  such  a  way  that  they  can  be 
transformed. 

Choose  problems  wisely.  Questions  that  trigger  a  change  of  mental 
model  should  be  carefully  selected  -  they  should  be  prototypical  of 
the  new  approach,  but  also,  they  should  be  just  beyond  the  trainees 
grasp.  They  should  not  be  full  of  distracting  cues  -  it  should  be  clear 
both  that  something  new  has  been  introduced  and  what  it  is. 

Using  multiple  learning  strategies  appears  to  be  beneficial.  Provision 
of  opportunities  for  (for  example)  open  ended  explorations, 
information  gathering,  problem  solving  practice,  demonstrations,  and 
so  on  can  support  this  approach. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 
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Description 

Thinking  and  consciously  makings  sense  of  information  when  reading 
or  attending  a  lecture  can  improve  learning.  Networking  is  another 
method  to  apply  to  this. 

How  to  do  it 

The  method  is  to  encourage  the  learner  to  think  about  how  different 
parts  of  the  information  fit  together,  how  it  all  makes  sense. 
Specifically,  the  learner  is  to  construct  a  network  in  which  information 
elements  are  explicitly  linked,  and  in  which  the  nature  and  content  of 
each  link  is  consciously  considered. 

Outlining  is  a  related  technique.  The  differences  between  the  two  are 
instructive.  The  basic  mode  in  outlining  is  to  group  and  differentiate, 
and  then  to  sequence.  Grouping  and  differentiation  tends  to  rely  on 
attributes  -  things  are  similar  because  they  have  similar 
characteristics.  Outlining  tends  to  be  heirarchical,  with  less  emphasis 
given  to  the  interconnections  among  the  diverse  sections. 

Networking,  by  contrast,  emphasizes  the  interconnections,  and  is 
especially  suited  to  domains  where  grouping  by  attribute  is 
misleading  and  the  underlying  causal  connections  are  a  critical 
element  of  the  subject  matter. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  simple  memory  tricks. 
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OPERATIONALIZE  DEFINITIONAL  CONCEPTS 

Description  t 

Complex  definitional  concepts,  such  as  are  taught  in  science  and 
engineering,  can  be  difficult  to  learn.  Operationalizing  the  concepts 
adds  both  understanding  and  retention. 

How  it  works 

Frederick  Reif  developed  a  sequence  of  four  steps  for  teaching 
complex  definitional  concepts: 

1.  Formulate  a  clear  definition  of  the  concept  and  accompany  it  by 
an  explicit  procedure  specifying  how  this  concept  can  be 
interpreted  in  any  particular  instance. 

2.  Let  students  practice  applying  this  definitional  procedure  to 
interpret  the  concept  in  a  variety  of  special  cases. 

3.  Let  students  use  their  explicit  knowledge  about  the  concept  to 
confront  situations  that  are  error-prone,  either  because  of 
intrinsic  difficulty  or  because  of  confusion  with  prior  knowledge. 
In  each  case,  ask  them  to  detect  mistakes  of  concept 
interpretation,  diagnose  the  likely  reasons  for  them,  and  correct 
them. 

4.  Guide  students  to  summarize  and  organize  their  accumulating 
case-specific  knowledge  so  as  to  acquire  a  useful  repertoire 
thereof— and  then  encourage  students  to  use  their  case-specific 
knowledge  in  familiar  situations. 

A  key  element  of  this  approach  is  having  the  learners  construct  and 
articulate  explanations,  notably  in  steps  3  and  4. 

Indicators 

Any  field  in  which  definitional  concepts  are  a  critical  element  in 
problem-solving. 

Counterindicators 

Domains  where  knowledge  is  empirical  or  procedural. 

Common  problems 

Students  may  resist  offering  explanations  that  might  reveal  lack  of 
knowledge  or  misunderstanding.  A  safe  learning  environment  is 
critical. 
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Related  techniques 

Self-explanation 

How  to  learn  complex  information 

References 

Reif  &  Allen  (1992) 


Syukhtun  Research,  Inc. 


239 


OVERLEARNING 


Description 

Overlearning  .happens  when  you  keep  training  something  that  the 
person  already  knows  how  to  do,  until  it  is  totally  ingrained.  This 
means  it  is  easier  to  do  and  attention  is  freed,  at  the  cost  of  being 
less  flexible. 

Memory  seems  to  be  based  on  cues  eliciting  memories,  and  memories 
eliciting  each  other.  Practicing  a  task  after  you  already  know  how  to 
do  it,  even  if  performance  does  not  improve,  will  still  lead  to  stronger 
and  stronger  associations  between  the  different  memories  used  in 
the  task,  and  between  the  external  cues  and  the  memories  they  elicit 
(which  lead  to  behavior).  Thus,  the  more  you  practice  it,  the  more 
strongly  the  cue  will  elicit  the  behavior,  and  in  a  sequence,  the  more 
strongly  each  behavior  will  elicit  the  next  -  what  is  called  a  strong 
memory. 

Overlearning  simplifies  complex  tasks  by  automating  complex 
responses.  An  overlearned  task  can  still  be  done  under  conditions 
that  do  not  allow  full  concentration  (stress).  Overlearning  also 
increases  the  sense  of  control,  predictability,  and  confidence. 

The  problem  is  that  overlearning  decreases  flexibility  -  both  in  that 
trainees  will  not  adjust  the  way  they  do  the  task  based  on  the 
environment,  and  that  they  won't  learn  from  what  they  are  doing. 
When  something  is  overlearned,  it  is  unlikely  that  the  trainee  is  going 
to  learn  anything  new  about  it  -  they  are  going  to  continue  to  do  the 
task  the  way  they  overlearned  it. 

How  to  do  it 

Teach  them  whatever  is  to  be  overlearned.  Then  have  them  practice 
more  even  after  they  can  do  it.  They  don't  necessarily  have  to  get 
better  at  it  to  be  advancing  if  the  goal  is  to  overlearn. 

If  possible,  avoid  overlearning  in  a  non-criterion  situation,  or  training 
in  ways  that  will  elicit  different  reactions  in  the  criterion  situation  than 
in  training.  These  will  lead  to  poor  training,  and  in  fact  can  impede 
performance. 
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One  important  factor  for  stress  is  confidence.  If  you  can  just  say  "I 
know  this  is  what  I  am  supposed  to  do/'  you  will  be  a  lot  more 
confident  in  doing  it,  and  in  deciding  to  do  it.  This  is  an  advantage  of 
overlearning  in  contrast  with  consciously  understanding  the  situation. 
If  you  understand  the  situation,  you  may  be  full  of  doubt  and 
conflicting  responses.  With  overlearning,  you  just  do  it.  Of  course,  it 
may  be  that  the  overlearned  response  is  inappropriate,  in  which  case 
training  overlearning  would  be  a  mistake. 

Don't  attempt  to  develop  overlearning  if  there  are  two  competing 
responses  triggered  by  the  same  stimulus.  Overlearning  competing 
responses  raises  the  possibility  of  choosing  the  wrong  one,  or  to 
freeze  because  two  dominant  responses  are  selected  at  the  same 
time. 

Actually  teach  variation  and  flexibility.  People  can  learn  to  look  for 
cues  and  make  calculations  as  part  of  an  overlearned  behavior,  if 
necessary,  although  if  flexibility  is  needed  overlearning  is  probably 
not  a  desirable  approach  unless  very  high  levels  of  stress  are  part  of 
the  task. 

For  generalization  to  occur,  make  sure  the  task  is  practiced  in 
different  situations,  so  that  the  practice  environment  does  not  come 
to  be  overly-associated  with  the  behaviors.  Said  another  way,  the 
behaviors  should  still  be  elicited  in  the  absence  of  the  training 
environment. 

Indicators 

Overlearning  is  most  useful  in  tasks  that  always  require  the  same 
response,  reactions  that  should  always  occur,  whatever  the  stress, 
distraction,  or  environment. 

Counterindicators 

Only  overlearn  things  that  are  easy  to  identify.  If  there  are 
competing  responses  triggered  by  the  same  cue,  or  if  a  cue  is 
misidentified,  trainees  may  not  adjust  their  behavior  appropriately  or 
know  what  to  do. 

The  overlearned  response  should  be  universally  adaptive.  A  response 
that  has  to  be  changed  as  the  event  progresses  should  not  be 
overlearned,  nor  should  responses  that  are  only  applicable  in  special 
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circumstances. 

Common  problems 

If  you  don't  get  it  right,  the  trainees  will  overlearn  the  wrong 
behavior. 

Boredom,  fatigue,  and  sloppiness  often  arise.  Avoid  fatigue  and 
boredom  while  practicing,  because  they  can  also  come  to  be 
associated  with  the  the  task. 

Related  techniques 

Automaticity 

Shaping 
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PAIRED  POST-TRAINING 

Description 

Trainees  use  learned  training  skills  on  each  other  when  training  is 
finished.  They  are  given  an  opportunity  to  observe  objectively  instead 
of  subjectively  when  a  skill  is  being  used.  It's  been  found  that  paired 
post-training  helps  ensure  good  training  behaviors  are  maintained 
and  transferred  to  actual  task  performance. 

How  to  do  it 

After  finishing  training,  pair  up  the  trainees.  Pairing  can  be  especially 
helpful  when  trainees  are  about  to  enter  the  work  environment.  Give 
some  attention  to  pairing  from  this  viewpoint.  It  can  also  be 
considered  an  aid  to  socialization. 

Tell  the  trainees  to  monitor  each  other  and  to  give  each  other  tips, 
feedback  and  reinforcement.  In  this  way,  training  habits  become 
better  established,  and  learning  opportunities  are  provided  for  the 
trainees  that  they  might  otherwise  miss.  In  a  sense,  you  let  them 
train  each  other.  By  alowing  them  to  practice  together  without 
supervision  they  will  also  gain  supervision  skills.  It  makes  sense, 
though,  to  provide  final  monitoring  of  tasks  assigned  to  pairs.  This 
helps  to  prevent  bad  habits  and  misunderstandings  being  passed 
from  one  partner  to  the  other.  For  this  reason,  too,  keep  paired 
training  sessions  short  to  begin  with.  Let  pairs  compare  results  with 
other  partnerships.  It  may  help  to  end  with  feedback  from  the  whole 
group  about  each  session  of  paired  post-training. 

Whenever  trainees  are  left  to  practice  unsupervised,  pairing  can  be 
used  in  the  same  way.  This  will  lay  the  ground  for  more  skilled  and 
directed  paired  post-training.  In  some  cases,  however,  it  is  probably 
more  helpful  to  emphasize  practice  alone  as  a  post-training  approach 
(see  counterindicators). 

Indicators 

This  technique  is  useful  after  training  when  the  trainee  enters  the 
task  environment.  It  will  help  to  insure  that  skills  transfer  takes  place, 
and  that  the  trainee  continues  to  learn  on  the  job.  Use  paired  post¬ 
training  when  the  job  involves  partnership  of  any  kind.  It  is  also 
useful  in  order  to  ease  socialization  and  to  help  job  entrants  feel 
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comfortable  asking  for  help  and  advice. 

Counterindicators 

If  training  has  involved  a  great  deal  of  pairing,  but  the  actual  skills 
are  meant  to  be  done  alone,  practice  alone  will  be  a  better  post¬ 
training  technique.  The  same  may  be  true  when  the  task  involves  a 
high  degree  of  concentration.  However,  paired  post-training  may  still 
be  useful  in  these  cases  when  trainees  have  difficulty  interpreting 
feedback  without  help. 

Common  problems 

Bad  habits  and  misinformation  can  be  catching.  Final  monitoring  and 
short  initial  sessions  should  be  in  place.  Personality  clashes,  too,  can 
make  paired  post-training  unsuccessful.  It  is  important  to  form  post¬ 
training  pairs  with  care.  This  will  help  to  avoid  pairing  partners  when 
one  does  not  feel  the  other  is  credible. 

The  idea  of  paired  training  itself  may  not  seem  credible  to  trainees. 
Using  the  technique  in  training  itself  prepares  the  ground.  Use 
related  techniques  to  overcome  resistance  to  socialization.  Assign 
specific  tasks  to  aid  monitoring  and  feedback. 

Pairs  will  vary  in  the  time  taken  to  accomplish  assignments.  Have 
further  tasks  ready  for  those  who  finish  quickly. 

Related  techniques 

Socialization 

Feedback  -  Action  Vs  Learning 

Feedback  -  Delay  It 

Feedback  -  Withhold  It 

Practice  Alone  (An  alternative  approach) 

Advance  And  Back  Off 
Behavioral  Modeling 
Lead  By  Example  -  Behavior 
Lead  By  Example  -  Atttitude 
Mentors  (An  alternative  approach) 
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PATH-GOAL  THEORY  (FOR  LEADERSHIP) 

Description 

A  trainer  motivates  by  defining  goals,  removing  obstacles  to  attaining 
them,  and  rewarding  their  attainment.  A  good  relationship  is  created 
between  trainer  and  trainees  when  a  clear  path  to  their  goal  and  its 
rewards  can  be  seen. 

How  to  do  it 

Neider  and  Schriensheim  (1988)  present  three  stages. 

1.  Precursor:  Job  analysis,  compensation  analysis,  employee 
selection,  orientation  training. 

2.  Maintenance:  Career  pathing,  goal  seting,  contingent  reward  and 
punishment,  "individualised  consideration." 

3.  Reassessment/monitoring:  Performance  feedback,  problem 
solving,  and  reassessment  of  subordinate  needs. 

Simplified,  they  suggest  that  managers  help  the  learner  to  set  goals, 
remove  obstacles  (as  far  as  is  appropriate),  let  them  know  that  they 
will  be  rewarded  if  they  acomplish  the  goals,  then  reward  them. 

Indicators 

A  long-term  relationship  with  the  trainee,  and  time  and  skill  to 
provide  the  guidance. 

Counterindicators 

This  method  requires  high  levels  of  supervision  and  long  term  follow- 
through.  The  leader  needs  to  be  able  to  control  things  like  obstacles 
and  rewards. 

Common  problems 

Dilbertian  management,  especially  a  mismatch  between  the  trainer 
and  the  trainee  on  skills  assessment,  goal  setting,  or  performance 
evaluation. 

Related  techniques 

Contract  theory 

Making  training  seem  important 
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PHYSICAL  GUIDANCE 

Description 

When  you  want  a  person  to  learn  to  do  a  physical  task  in  a  certain 
way,  a  good  approach  can  be  to  guide  their  movements.  In  this  way, 
the  trainee  learns  what  it  feels  like  to  do  it  right. 

How  to  do  it 

There  are  two  approaches  -  forced  response  and  restriction. 
Restriction  is  when  you  only  allow  the  trainee  to  make  the  correct 
movements.  Forced  response  is  when,  for  example,  you  hold  their 
arm  and  move  it  the  way  it  needs  to  be  moved.  These  methods  make 
it  clear  which  physical  movements  and  positions  are  needed. 
Restriction  is  generally  preferable,  but  harder  to  do.  It  calls  for  close 
supervision,  constant  repetition,  and  flexibility  when  the  approach 
fails.  Forced  response  can  be  a  support  approach  to  restriction  since 
it  will  sometimes  be  the  fastest  way  to  help  a  trainee  understand 
what's  required.  Prior  demonstration  is  very  important  to  both 
methods. 

An  ideal  use  of  physical  guidance  is  to  set  up  initial  positions.  Once  in 
the  correct  position,  a  trainee  can  often  perform  the  skill  being  taught 
without  further  use  of  the  technique. 

Allow  variation  from  ideal  movements.  This  will  create  feedback 
showing  when  a  movement  is  wrong,  and  also  provide  contrast  which 
will  help  show  which  movement  or  position  is  right.  Support  the  set  of 
movements  required  with  an  explanation  and  description.  This  will 
help  trainees  to  correct  their  form  objectively  when  confusion  arises 
during  the  movements  themselves. 

Always  follow  physical  guidance  with  practice  alone  and  monitoring. 
This  allows  the  trainees  to  work  from  feedback  and  ensures  mistakes 
are  picked  up. 

Indicators 

This  technique  is  used  in  training  physical  movements.  It  is  especially 
useful  when  a  trainee  fails  to  master  a  skill  after  repeated  feedback. 
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Counterindicators 

In  some  situations,  physical  guidance  will  take  more  effort  than 
explaining  and  repeated  demonstrations.  In  large  groups  in 
particular,  this  kind  of  one-to-one  guidance  may  not  be  feasible. 

If  trainees  resist  when  you  guide  them,  they  may  learn  to  do  the 
action  incorrectly.  Their  independent  movements  will  tend  to  revert  to 
the  position  you  were  guiding  away  from.  In  this  case  physical 
guidance  is  inappropriate. 

Common  problems 

Some  trainees  will  resist.  Explanation  and  demonstration  may  prove 
better  techniques  in  these  cases.  However,  restriction  technique 
alone  may  be  sufficient. 

In  large  groups,  alternative  training  techniques  may  need  to  be  used 
to  keep  other  trainees  occupied  while  physical  guidance  is  being  used 
on  a  one-to-one  basis. 

Related  techniques 

Limit  Introduction 
Learn  By  Doing  Only' 

Immediate  Practice 
Automaticity 
Feedback  Sections 
Behavioral  Modeling 
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PLANNED  CUES  FOR  MEMORY 

Description 

If  you  want  to  remember  something  at  a  certain  time  or  in  a  certain 
place,  you  can  form  an  association  between  the  thing  you  want  to 
remember  and  something  you  are  going  to  do  or  encounter  at  either 
that  time  or  place. 

How  to  do  it 

If  you  want  to  remember  something  at  a  certain  place,  form  a  mental 
image  that  will  remind  you  of  what  you  want  to  remember  at  that 
place.  For  example,  when  you  get  home,  remember  to  call  the 
doctor,  so  you  imagine  the  doctor  on  your  doorstep. 

Make  it  specific.  Not  "have  lunch/'  but  "open  the  refrigerator  door." 
Lunch  is  not  really  a  specific  cue,  whereas  the  fridge  door  is. 

This  technique  works  best  if  there  is  some  cue  in  the  world  that  will 
trigger  the  memory.  If  you  have  to  remember  something  at  a  certain 
time,  find  something  that  will  be  happening  at  that  time  to  use  as  a 
cue.  The  more  certain  it  is  that  you  will  notice  the  cue,  the  better  the 
likelihood  that  the  memory  will  be  triggered. 

This  technique  is  effective  for  near-term  recall,  but  there  is  no 
research  suggesting  it  for  long-term  memorization. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 
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Description 

PQ4R  stands  for  Preview,  Question,  Read,  Reflect,  Recite,  Review. 

This  is  a  standard  technique  to  encourage  thorough  understanding  of 
textual  material. 

Mow  to  do  it 

The  method  is  as  it  sounds:  The  learner  first  previews  and  generates 
questions  about  the  material;  reads  it  carefully;  reflect  on  its  meaning 
while  it  is  being  read;  recites  as  much  of  the  material  as  possible; 
and  finally,  reviews  the  material,  using  the  questions  developed 
before  begininniog  reading. 

This  is  a  useful  method,  but  it  lacks  criteria  for  applicability.  For  the 
learner,  there  is  a  cost/benefit  ratio  that  is  constantly  being 
unconsciously  assessed.  PQ4R  is  a  lot  of  work,  and  if  the  perceived 
benefit  isn't  adequate  the  learner  will  quickly  abandon  it.  Recitation  in 
particular  is  unlikely  to  be  popular,  especially  since  the  empirical 
evidence  supporting  its  effect  on  learning  and  retention  is  less  than 
compelling. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 
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PRACTICE  ALONE 


Description 

Observing  a  trainee  during  practice  will  make  the  practice  into 
something  of  a  performance  in  itself.  A  person  trying  to  practice  with 
another  person  in  the  room  tends  to  have  less  ability  to  monitor  what 
he  or  she  is  doing.  The  performance  beomes  monitored  rather  than 
skills.  Practice  alone  occurs  without  an  observer  and  leads  to  more 
ability  to  concentrate,  and  therefore  more  efficient  practice. 

How  to  do  it 

Give  the  trainee  an  assignment  and  leave  the  room.  Research  shows 
that  the  presence  of  another  person,  even  one  who  is  paying  no 
attention  at  all,  can  effect  concentration.  The  presence  of  a  trainer  - 
the  person  whose  opinion  counts  the  most  -  may  have  an  especially 
strong  effect. 

Following  sessions  where  the  trainer  teaches  new  skills  and  provides 
feedback  and  reinforcement,  the  trainee  will  simply  need  practice. 

This  is  then  achieved  best  alone.  It  can  be  difficult  to  judge  when 
trainees  have  enough  information  both  to  make  advances  and  to 
accurately  monitor  their  performance.  Nevertheless,  it  is  important  to 
allow  frequent  opportunities  for  practice  alone.  It  is  equally  important 
to  also  occasionally  use  supervised  practice  to  ensure  wrong  skills 
are  not  unintentionally  being  learned  during  practice  alone. 

If  skills  are  going  to  be  used  around  others,  observed  practice  should 
also  be  used  once  trainees  have  reached  a  satisfactory  standard.  In 
this  way,  trainees  become  used  to  performing  or  simply  working  in 
front  of  other  people.  This  is  necessary  in  order  to  ensure  that  skills 
learned  transfer  to  the  actual  environment  where  they  will  be  used. 

Indicators 

Practice  alone  is  useful  for  people  who  are  practicing  something  that 
requires  concentration  and  that  has  observable  consequences.  It  is 
especially  applicable  when  the  skills  are  oriented  to  performance 
where  mistakes  are  highly  visible  and  detrimental  to  the  value  of 
performance. 
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Counterindicators 

Practice  alone  is  unnecessary  when  a  trainee  has  reached 
performance  standard.  In  this  case,  performance  itself  can  increase 
drive  and  standards  so  an  observer  will  lead  to  training  benefits.  In 
addition,  the  narrowed  attention  resulting  from  observed  practice 
may  suit  simple  tasks  which  do  not  require  the  trainee's  full  mental 
resources.  Less  motivated  individuals  will  also  accomplish  less  alone. 
They  will  do  better  in  supervised  practice. 

Common  problems 

Without  feedback  there  is  the  possibility  that  bad  habits  will  become 
established  with  unobserved  practice.  Use  a  balance  of  practice  alone 
with  supervised  sessions. 

Motivation  may  lag  when  the  person  is  practicing  alone.  Again, 
practice  alone  combined  with  other  training  techniques  should  lessen 
this  problem. 

Related  techniques 

Deliberate  Practice 
Covert  practice 
Learn  by  doing  only 
Limit  introduction 
Instruction 

Establish  an  environment 
Feedback  -  Action  vs  Learning 
Advance  and  back  off 

References 

Orasanu  and  Backer 
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RECODING 

Description 

Information  that  is  bland  and  uninteresting  (such  as  a  set  of  numbers 
or  letters)  can  be  recoded  in  a  more  accessible  way. 

How  to  do  it 

The  to-be-remembered  item  is  recoded  in  a  more  accessible  way. 

For  example,  the  phone  comes  with  letters  to  go  along  with  the 
numbers.  A  random  set  of  numbers  is  hard  to  remember.  So  make 
words  out  of  the  letters,  because  words  are  a  lot  easier  to  remember 
than  letters,  and  you  can  associate  the  word  with  the  person  you 
want  to  call.  Or,  you  can  think  of  the  numbers  as  race  times,  baseball 
statistics,  or  birthdays.  Or,  you  can  recode  the  numbers  in  terms  of 
number  facts,  e.g.,  if  the  phone  number  is  777-3610,  it  is  three  7's, 
then  3,  then  3  +  3,  then  3+3+4. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  complex  memory 
tricks. 

References 

Kyllonen  &  Alluisi 
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SELF-EXPLANATION 

Description 

Generating  explanations  for  yourself  as  you  read  boosts 
comprehension  of  complex  material. 

How  it  works 

Chi  et  al  (1994)  found  that  comprehension  was  boosted  significantly 
by  generating  explanations  for  read  material.  Their  experiment  had 
high  school  students  read  a  selection  from  a  popular  biology  text.  The 
control  group  read  it  through  twice.  The  test  group  read  it  once,  but 
(as  instructed)  stopped  after  each  sentence  and  generated  an 
explanation  for  themselves  about  what  it  meant.  The  explainers 
understood  the  passage  much  better  than  the  twice-readers. 

Especially  interesting,  one  of  the  Chi  team's  results  was  that  the  self¬ 
explainers  frequently  generated  false  hypotheses,  but  that  these 
were  self-correcting.  Apparently  because  they  were  self-articulated, 
the  students  took  pains  to  check  and  correct  them.  In  effect,  they 
used  hypotheses  as  they  are  supposed  to  be  used,  as  mental  tools 
to  guide  investigation. 

Indicators 

Any  situation  involving  reading,  understanding,  and  retaining  complex 
information. 

Counterindicators 

Tasks  that  don't  involve  reading  complex  information. 

Common  problems 

It  takes  work  and  requires  breaking  long-standing  habits. 

Related  techniques 

Mental  simulation 

References 

Chi  et  al 
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SEQUENCING:  COMPLETE  OR  SIMPLIFIED 

Description 

This  approach  is  a  choice  between  teaching  the  fundamentals  of  a 
task  first  or  allowing  trainees  to  start  immediately  on  the  full  task. 
Tackling  basics,  task  components,  or  simplified  versions  of  the  task 
can  be  time  consuming  and  boring,  but  may  be  easier  in  the  short 
term,  and  produce  more  thorough,  longer  lasting  results.  Practicing 
the  actual  task  may  allow  the  trainee  to  develop  whatever 
fundamentals  are  necessary  while  actually  doing  it.  More  effort  may 
be  required  at  first,  but  greater  challenge  is  provided.  Whichever  is 
chosen  first,  the  two  approaches  can  be  combined  in  a  flexible 
sequence  to  meet  trainees'  developing  needs. 

How  to  do  it 

Simplified  task  first:  The  most  basic  method  of  simplification  is  to 
break  the  task  down  into  its  components.  Avoid  training  isolated  skills 
in  ways  which  are  inappropriate  for  the  actual  task.  It  is  useful  to 
begin  with  skills  which  easily  become  automatic.  This  frees  up 
cognitive  resources  to  be  used  during  the  task,  and  increases  speed 
and  accuracy  (see  Automaticity). 

A  simplified  version  of  the  whole  task  can  also  be  created.  This  can  be 
done  in  several  ways:  Remove  elements  from  it  (see  Adaptive 
Training);  simplify  the  required  responses  while  leaving  the  task  the 
same;  change  the  required  responses.  The  danger  with  simplification 
is  that  trainees  may  learn  to  carry  out  actions  in  ways  which  are 
unsuitable  to  the  actual  task.  In  the  long  term  this  may  mean  the  full 
task  is  harder  to  teach.  In  the  meantime,  however,  a  through 
understanding  of  the  basics  involved  will  hav  been  gained  without 
overwhelming  the  trainees. 

Simplification  is  improved  if  instructions,  guidance,  safety  nets, 
feedback,  and  reinforcement  are  built  in  for  every  step.  You  may  want 
to  focus  particularly  on  the  most  difficult  parts,  actually  segmenting 
the  task  and  applying  a  full  range  of  training  techniques  at  every 
stage. 

Train  without  time  stress.  Time  stress  causes  trainees  to  simplify  their 
approach  and  ignore  information  in  order  to  make  the  task  easier. 
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(This  may  be  appropriate  when  sequencing  the  complete  task  first.) 

If  the  task  requires  conceptual  knowledge,  mental  models,  strategies 
and  other  things  that  necessitate  cognitive  processes  during  the 
task,  these  can  be  taught  explicitly  to  the  student  near  the  beginning 
of  the  training.  As  training  becomes  more  complicated,  the  trainees 
will  be  able  to  use  their  basic  understanding,  and  to  practice  the 
cognitive  skills  they  will  use  in  the  task.  However,  if  the  task  requires 
doing  an  action  faster  than  these  reasoning  strategies  allows,  teach 
a  heuristic  approach  instead  (see  Don't  Take  Your  Time  -  Heuristics). 

In  this  way,  trainees  will  develop  the  necessary  cognitive  processes 
reflectively,  in  tandem  with  training. 

Motivation  should  be  monitored  when  using  simplified  first.  Trainees 
can  become  discouraged  if  the  complete  task  goal  is  too  far  off.  Use 
other  training  techniques  to  maintain  a  positive  mood,  establish  an 
environment  and  use  contract  theory. 

Complete  task  first:  Start  with  the  full-on,  hard  version  while  still 
maintaining  standard  training  such  as  feedback  and  new  skill 
teaching.  This  approach  can  be  supplemented  with  others  such  as 
training  with  a  timer  and,  as  with  simplified  tasks,  heuristics. 

It  is  important  to  monitor  whether  skills  fail  in  any  particular  area. 
When  this  occurs,  be  prepared  to  switch  to  simplified  task  teaching. 
One  advantage  of  sequencing  complete  tasks  first  is  that  it  can 
reveal  where  trainee  skills  are  weak  and  enable  training  to  adjust 
accordingly. 

Using  complete  task  training  first,  it  is  likely  that  you  will  find  that  the 
trainee  is  able  to  make  a  lot  of  progress  for  a  while,  but  will  slow 
down.  This  contrasts  with  those  who  are  started  on  fundamentals 
who  will  find  doing  the  whole  task  relatively  difficult  at  first,  but  will 
have  the  capacity  to  improve. 

Indicators 

It  makes  sense  to  sequence  the  complete  task  first  when  the  task  is 
simple,  or  only  to  be  performed  a  few  times.  In  addition,  some  tasks 
do  not  contain  fundamentals  which  could  be  taught  as  components  or 
in  simplified  versions.  Other  tasks  may  not  need  to  be  performed 
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perfectly  or  with  full  understanding  and  are  therefore  suitable  to 
teach  in  their  complete  state.  Sometimes  time  constraints  may  lead  to 
the  complete  task  being  taught  first.  Some  trainees  may  respond  well 
to  challenges,  and  would  therefore  benefit  from  complete  task 
training. 

Sequencing  simplified  task  teaching  first  is  suitable  when  the  ultimate 
performance  standard  is  very  distant,  the  task  is  very  complex,  or  the 
task  is  hard  to  understand.  If  the  components  of  a  task  are  not  highly 
interdependent,  and  they  employ  different  skills,  simplified  first  will 
probably  be  the  optimum  sequence.  If  there  are  elements  of  the  task 
that  the  person  is  likely  to  learn  incorrectly,  never  learn,  or  not  learn 
optimally  when  carrying  out  the  full  task,  these  are  suitable  for  a 
simplified  approach.  Because  a  thorough,  automatic  and  lasting 
understanding  is  gained  of  basic  components,  simplified  first 
sequencing  is  appropriate  when  the  long  term  aim  is  for  the  task  to 
be  completed  efficiently  and  fast. 

Counterindicators 

This  is  mainly  covered  by  the  factors  leading  to  a  choice  of  one 
sequence  over  the  other.  Be  prepared  to  switch  from  one  task 
training  approach  to  the  other  as  trainee  skill  levels  and 
understanding  indicate.  It  should  also  be  noted  that  the  cost  of 
training  on  some  complete  tasks  may  be  prohibitive. 

Common  problems 

Changing  the  task  to  make  it  easier  may  make  it  boring.  Simplifying 
the  task  can  also  make  it  harder  for  trainees  to  advance.  It's  been 
shown  that  people  learn  more  easily  about  new  activities  than  ones 
to  which  they  are  accustomed.  Be  prepared  with  many  versions  of 
varying  difficulty  and  different  approaches.  Monitor  mood  and  use 
t  training  techniques  such  as  establishing  an  environment  and 
socialization. 

If  you  sequence  the  complete  task  first,  monitor  trainee 
understanding.  The  task  itself  may  be  challenging  and  stimulate  fast 
learning  of  several  things  at  once.  However,  it  may  also  lead  to  an 
artificially  simplified  and  incomplete  understanding.  Again,  monitoring 
and  flexibility  between  the  two  sequences  are  key. 

Related  Techniques 
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Automaticity 

Learn  by  mucking  about 

Train  with  a  timer 

Don't  take  your  time:  heuristics 

Overlearning 

References 

Holding 

Glaser  and  Bassok 
Salas,  Driskell,  Hughes 
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SHAPING 

Description 

Shaping  is  a  training  approach  based  on  reinforcement  of  successive 
approximations  to  the  desired  behavior.  Shaping  is  good  for 
development  of  behavioral  responses  to  perceived  circumstances. 
"Behavioral"  is  placed  in  mild  contrast  here  with  "mental,"  since 
conscious  awareness  of  the  shaping  process,  or  even  of  the 
contingency  of  the  reinforcement  on  the  behavior,  is  not  required  for 
shaping  to  work.  Shaping  is  based  on  the  principle  that  actions  that 
are  rewarded  are  more  likely  to  occur  in  the  future. 

How  to  do  it 

Shaping  works  through  reinforcement.  Reinforcement  can  be  anything 
a  trainee  wants.  Reinforcement  is  like  reward,  with  the  crucial 
distinction  that  it  only  counts  as  reinforcement  if  (a)  it  happens  in 
close  conjunction  with  an  act,  and  (b)  it  increases  (positive 
reinforcement)  or  decreases  (negative  reinforcement)  the  likelihood 
of  the  act  occurring  again. 

Both  elements  of  this  definition  count.  First,  conjunction  with  the  act 
is  important.  The  greater  the  distance  in  time  (or  the  greater  the 
psychological  distance),  the  weaker  the  shaping  effect.  When  the 
receiver  makes  the  touchdown  catch  and  the  stadium  explodes, 
that's  positive  reinforcement.  When  the  grant  is  awarded  fourteen 
months  after  the  grant  proposal  was  submitted,  that's  reward,  but 
it's  not  reinforcement. 

Second,  in  shaping,  "positive"  and  "negative"  reinforcements  are 
defined  solely  in  terms  of  the  likelihood  of  the  target  action  recurring. 
They  are  not  defined  in  terms  of  emotional  qualities.  For  example,  if  a 
lonely,  neglected  child  cannot  get  attention  in  any  other  way,  then 
the  scolding  that  follows  misbehavior  is  positive  reinforcement.  The 
child  has  found  a  way  to  get  attention,  something  he  or  she  wants. 
The  emotionally  negative  scolding  is  actually  a  strong  positive 
reinforcement. 

"Successive  approximation"  assumes  you  aren't  starting  out  with  the 
behavior  you  want.  To  shape  the  behavior,  movement  in  the  direction 
of  the  desired  behavior— "approximation"— is  reinforced.  Learning 
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proceeds  in  steps,  with  each  approximation  growing  closer  to  the 
desired  behavior.  For  this  to  happen,  the  learner  must  vary  in 
behavior.  The  learner  must  try  things,  experiment  and  explore,  so 
that  the  shaper  can  reinforce  successively  closer  approximations. 

The  basic  process  of  shaping  is  to  reinforce  closer  and  closer 
approximations  of  the  desired  behavior,  so  as  to  eventually  arrive  at 
the  desired  performance.  There  are  ten  "laws"  of  shaping  (from 
Karen  Pryor): 

1.  Raise  criteria  in  increments  small  enough  that  the  trainee  has  a 
reasonable  chance  for  reinforcement. 

You  want  the  trainee  to  succeed.  How  fast  you  "raise  the  bar"  is  not 
a  function  of  the  potential  you  perceive  in  the  trainee.  Instead,  it 
depends  on  how  well  you  communicate  through  the  shaping 
procedure  what  patterns  of  behavior  get  reinforcement.  It's  useful  to 
note  here  that  "communication"  should  not  be  confused  with  the 
words  spoken  by  the  trainer.  Some  psychologists  like  to  use  the 
metaphor  of  "the  words  and  the  music".  The  words  are  part  of  the 
message,  but  people  are  normally  highly  sensitive  to  the  way  the 
words  are  spoken  (the  "music").  One  person  can  say  "that  was  OK" 
and  make  it  a  powerful  positive  reinforcement.  Another  can  say  "that 
was  great"  and  turn  it  into  a  strong  negative  reinforcement.  In 
general,  the  music  counts  much  more  than  the  words. 

2.  Train  one  thing  at  a  time. 

This  doesn't  mean  one  task  at  a  time.  It  simply  means  that  if  you  are 
training  a  particular  subtask,  work  on  only  one  aspect  of  it  at  a  time. 
If  you  are  training  the  drive  off  the  tee  in  golf,  and  you  are  working 
on  the  follow-through,  then  just  work  on  the  follow-through  until  it  is 
mastered.  However  the  subgoals  or  processes  of  a  task  are  broken 
out,  training  should  approach  them  one  at  a  time. 

3.  Always  put  the  current  response  on  a  variable  schedule  of 
reinforcement  before  adding  or  raising  criteria. 

"Variable  schedule"  simply  means  that  once  the  response  is  learned, 
the  reinforcement  does  not  always  automatically  follow.  Variable 
reinforcement  has  been  shown  repeatedly  to  strengthen  the  learned 
response.  When  you  can  safely  reinforce  only  some  of  the  time  and 
still  get  the  right  behavior,  you  can  then  reinforce  only  the  best 
examples  of  the  behavior. 
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4.  When  introducing  new  criteria,  relax  the  old  ones. 

This  is  similar  in  spirit  to  only  doing  one  thing  at  a  time.  When  there  is 
a  new  subtask  to  be  learned,  then  concentrate  on  it.  Make 
integrating  it  with  previously  learned  subtasks  a  distinct  and 
subsequent  learning  step. 

5.  Stay  ahead  of  your  trainee. 

This  means  that  if  the  trainee  makes  sudden  progress,  or  even  a 
breakthrough,  you  know  what  to  reinforce  next. 

6.  Don't  change  trainers  in  midstream. 

This  does  not  mean  there  cannot  be  different  trainers  for  different 
behaviors  working  at  the  same  (general)  time.  It  does  mean  that  the 
relationship  between  the  trainer  and  the  trainee  should  be  respected 
and  not  disrupted. 

7.  If  one  shaping  approach  is  not  working,  try  another  one. 

While  this  sounds  pretty  straightforward,  it  can  happen  that  a  trainer 
develops  a  belief  in  his  or  her  "best"  way  of  training  and  blames  the 
trainee  for  failing  to  learn.  Shaping  is  a  highly  cooperative  venture, 
and  if  the  trainer  is  not  seeing  progress,  the  right  thing  to  do  is  to  try 
a  different  approach,  a  different  breakout  of  subtask  elements,  or  a 
different  reinforcement. 

8.  Don't  interrupt  a  training  session  gratuitously. 

If  the  trainee  is  expected  to  give  his  or  her  full  attention  to  the 
training  session,  then  the  trainer  should  be  at  least  as  committed. 
Removal  of  attention  is  a  powerful  rebuke.  If  a  rebuke  is  appropriate, 
attention  removal  can  be  used,  but  if  it  is  used  carelessly,  e.g.,  to 
answer  a  telephone  in  the  middle  of  a  session,  the  trainee  has 
learned  that  the  trainer  can't  be  trusted  in  his  or  her  committment  to 
the  training.  The  training  must  not  be  very  important,  then. 

9.  If  behavior  deteriorates,  go  back  to  the  basics  and  provide  easy 
reinforcements. 

If  for  example  the  learner  gets  frustrated  learning  to  juggle,  then  you 
reinforce  her  for  just  trowing  the  balls  up,  not  too  hard,  and  gradually 
build  the  behavior  back  up,  and  along  with  it,  her  confidence.  Note 
that  you  are  rewarding  the  subject  for  performing  some  behavior,  but 
you  must  also  make  sure  the  learner  is  rewarded  for  participating  in 
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the  learning  process  itself. 

10.  End  every  training  session  on  a  high  note. 

If  necessary,  stop  earlier  than  planned  to  do  this.  If  scheduling 
problems  make  stopping  early  impossible,  or  if  fatigue  starts  to 
interfere  with  performance,  have  the  last  section  of  the  training 
session  focus  on  a  high-confidence  activity,  one  that  is  sure  to  garner 
reinforcement.  The  concomitant  of  this  is,  never  introduce  new  and 
difficult  tasks  at  the  end  of  a  training  session.  That  will  simply  train  a 
feeling  of  inadequacy  in  association  with  the  training  in  general. 

Measures  of  effectiveness  for  shaping  are  typically  simple,  i.e.,  the 
target  level  of  task  performance.  Because  the  first  part  of  shaping  is 
to  "disassemble"  the  task,  setting  up  MOEs  can  take  advantage  of 
this  preparatory  work.  MOEs  of  course  need  to  respect  normal 
performance  limits  of  the  trainees.  No  amount  of  brilliant  shaping  will 
get  someone  to  run  100  meters  in  five  seconds  or  juggle  thirty  clubs. 

The  cost  of  shaping  is  typically  more  in  the  form  of  ingenuity  and  hard 
work  on  the  part  of  the  trainer,  rather  than  in  dollar  costs  associated 
with  development  of  curricula,  simulations,  or  training  systems.  There 
are  no  particular  overhead  costs  with  shaping.  Shaping  is  a  low- 
dollar,  high-mental-investment  approach. 

Indicators 

Shaping,  carefully  applied,  can  be  extremely  powerful  in  modifying 
behavior.  It  has  even  been  used  to  train  pigeons  to  play  ping-pong. 

It  is  a  technique  for  developing  reliable  performance  of  complex 
behaviors.  In  particular,  if  a  pattern  of  behavior  requires  intelligence, 
but  not  necessarily  conscious  control,  shaping  is  an  appropriate 
technique.  Examples  of  "intelligent  but  not  consciously  controlled" 
behavior  include  real-time  performance  (sports,  some  aspects  of 
firefighting,  some  military  activities),  and  performance  where 
motivational  or  attitudinal  aspects  are  crucial,  e.g.,  meeting  new 
people  and  making  them  feel  at  ease. 

Counterindicators 

Shaping  is  an  inefficient  approach  if  the  range  of  possible  responses 
is  both  wide  and  internally  dissimilar,  or  if  a  great  deal  of  conscious 
control  is  required  to  perform  a  task.  Examples  of  the  former  include 
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design  tasks  where  there  are  relatively  few  constraints,  and  most 
artistic  composition  tasks.  Examples  of  the  latter  are  scientific, 
mathematical,  or  technical  problem-solving  skills.  However,  note  that 
in  both  of  these  cases,  while  shaping  would  not  be  useful  as  a 
primary  technique,  it  could  be  useful  as  a  supporting  technique. 

While  shaping  is  almost  always  applicable  to  physical  skills,  it  is  not 
the  approach  of  choice  for  problem-solving  skills.  Nor  is  it  the  choice  if 
there  are  explicit  rules  or  procedures  that  can  be  consciously  followed 
to  achieve  the  desired  goal. 

Shaping  is  not  for  the  intellectually  or  attentionally  lazy  trainer.  A 
fairly  high  skill  level  is  needed  to  be  effective.  Instructors  need  to 
exercise  their  analytical  sides  to  plan  shaping,  and  their  intuitive 
sides  to  carry  it  out.  Instructors  must  be  able  to  pay  very  close 
attention  to  the  trainees,  and  to  fully  engage  in  working  with  them. 
There  are  four  basic  skills  or  requirements: 

1.  Know  exactly  what  you  want,  in  whatever  level  of  detail  is 
necessary. 

2.  Pay  very  close  attention. 

3.  Know  what  is  good  enough  to  be  reinforced,  and  what  isn't,  that 
is,  at  what  level  to  "put  the  bar." 

4.  Know  what  is  reinforcing  for  the  individual  learner. 

Common  problems 

Planning  problems  occur  when  the  breakdown  of  the  task  to  be 
performed  does  not  correspond  to  distinct  behaviors,  and  when  the 
plan  is  incomplete.  Implementation  problems  can  stem  from  trainer 
rigidity  ("this  is  how  we  do  it,  goddamit"),  inadequate  reinforcement, 
over- reinforcement  (the  general  rule  is,  use  the  smallest  effective 
reinforcement),  and  impatience. 

A  special  class  of  implementation  problems  can  arise  when  the  trainer 
(unintentionally)  confuses  trying  with  goal  achievement.  It  may  be 
the  case  that  "trying"  is  the  target  behavior,  in  which  case 
reinforcement  of  trying  is  good  shaping.  This  sometimes  occurs  when 
the  correct  result  is  self-reinforcing  (like  in  juggling,  for  instance),  and 
the  learner's  hurdle  is  to  learn  to  try.  However,  if  the  desired 
performance  is  is  not  strongly  self-reinforcing,  then  reinforcing  trying 
only  trains  the  student  to  try,  but  not  necessarily  to  succeed. 
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Related  techniques  for  the  same  training  problem(s) 

Pure  reinforcement  is  similar  to  shaping,  with  the  difference  that 
approximations  to  the  goal  behavior  are  not  reinforced,  only  the  goal 
behavior  itself. 

Classroom  lecture  is  an  inverse  approach.  Good  teachers  actually  use 
shaping  techniques  (often  unconsciously).  The  classroom  setting, 
though,  and  the  classical  classroom  approach  of  standardized 
information  being  provided  verbally  to  a  room  full  of  students,  is  the 
opposite  of  shaping. 

Unsupervised  exploration  ("sink  or  swim")  is  the  trainerless  version  of 
shaping.  In  "sink  or  swim",  the  trainee  becomes  his  or  her  own 
trainer,  monitoring  the  environment  for  cues  as  to  progress,  with  the 
perception  of  a  step  forward  being  the  reinforcement  for  whatever 
action  led  to  it.  This  kind  of  training  often  leads  to  real  behavioral 
inefficiencies,  since  random  actions  that  happen  to  correlate  with 
success  become  ingrained.  Baseball  pitchers  who  have  a  ritualized 
nineteen-step  routine  before  every  pitch  display  this  kind  of  behavior. 
If  you  adjust  the  brim  of  your  hat  immediately  before  a  good  pitch, 
you're  more  likely  to  adjust  the  brim  of  your  hat  before  every  pitch.  In 
realtime  domains  such  as  emergency  medicine,  unnecessary  actions 
are  undesirable. 

In  some  cases  inefficiency  will  simply  need  to  be  accepted.  We 
animals  have  a  tendency  to  stick  with  what  works,  even  if  it  isn't 
perfect  (Herbert  Simon  won  a  Nobel  Prize  in  Economics  for  pointing 
this  out.)  If  the  learner  develops  a  pattern  that  leads  to  the  desired 
behaviors  but  includes  a  few  idiosyncracies,  well,  the  learner  is  never 
the  same  as  the  teacher. 

Unsupervised  exploration  should  not  be  confused  with  task 
immersion.  Task  immersion  occcurs  when  the  learner  has  achieved 
sufficient  grounding  in  the  task  to  be  placed  in  the  task  environment 
and  allowed  to  actually  perfrom  the  task.  Task  immersion  is  often 
structured  to  provide  exposure  to  the  easy  or  moderate  aspects  of 
the  task  in  a  graded  fashion,  and  is  rarely  used  when  there  will  not 
be  supervision  and  support  available  if  needed.  Nor  is  it  used  when 
there  isn't  already  evidence  that  the  learner  will  be  able  to  perform 
the  desired  task(s).  "Sink  or  swim"  is  task  immersion  without 
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safeguards  or  evidence,  which  makes  it  a  different  thing  entirely. 
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SOCIALIZATION 

Description 

Socialization  occurs  naturally  in  a  new  training  or  work  environment. 
New  entrants  automatically  adopt  prevailing  attititudes,  moods,  and 
expectations.  Socialization  as  a  training  technique  uses  the 
processes  involved  so  they  become  an  aid  to  more  effective  work  or 
training.  Socialization  is  especially  useful  as  training  or  work  begin. 

How  to  do  it 

The  effects  of  socialization  are  thought  to  be  continuous.  However, 
socialization  most  strongly  influences  work  or  training  at  the 
beginning.  Socialization  effects  make  initial  encounters  very 
important.  This  is  true  in  totally  new  environments,  but  it  is  also  seen 
when  entering  a  new  department,  starting  with  a  new  partner, 
entering  a  new  project,  or  whenever  the  work  or  training 
environment  changes.  Use  socialization  in  these  situations  because 
attitudes  are  more  open  and  more  easily  influenced  than  usual. 

It's  been  shown  that  people  are  born  conformists  and  learn  well  from 
social  cues  and  patterns.  Socialization  is  a  conscious  effort  to  direct 
these  natural  and  powerful  tendencies  in  a  constructive  manner.  You 
can  help  make  socialization  positive  by  first  ensuring  the  desirable 
attitudes,  moods  and  expectations  are  clearly  and  easily  perceived. 
Make  an  effort  to  create  a  good  first  impression.  It  helps  to  provide 
guidance  so  that  the  trainee  feels  comfortable,  confident,  and 
effective.  Ensure  the  work  or  training  is  already  running  in  a  way 
which  rewards  the  type  of  behaviour  you  would  like  new  entrants  to 
adopt. 

Specific  supplementary  techniques  to  use  are  the  buddy  system, 
organized  support  groups,  and  mentors.  These  approaches 
potentially  help  with  transfer,  because,  when  buddies,  groups  and 
mentors  are  chosen  with  care,  they  produce  exchanges  which 
reinforce  learnings. 

One  important  aim  is  to  establish  a  climate  where  learning  and 
training  are  encouraged  and  considered  important.  This  is  easily 
demonstrated  if  others,  including  the  trainer  or  manager  are  also 
undergoing  training  and  are  seen  to  take  it  seriously.  Leading  by 
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example  will  also  contribute  to  positive  socialization. 

Indicators 

Use  socialization  when  someone  is  first  entering  a  new  environment, 
or  if  the  environment  is  changing.  Also  use  it  if  there  is  a  strong  need 
for  esprit  du  corps. 

Counterindicators 

If  long  term  attitudes  are  not  involved  you  may  not  want  to  put  effort 
into  socialization.  However,  socialization  can  have  an  effect  even  on 
short  training  sessions  and  temporary  jobs.  It  can  be  adequately 
accomplished  by  the  existence  alone  of  a  successful  working  group. 

Common  problems 

So  much  effort  can  go  into  socialization  that  it  provides  a  false 
impression.  Credibility  may  be  lost,  and  disappointment  can  set  in 
when  the  reality  is  discovered.  If  negative  situations  exist, 
socialization  can  partly  be  based  on  positive  and  honest  attitudes 
etc.  about  resolving  the  problems  involved. 

Related  techniques 

Expectations  of  training 
Establish  an  environment 
Choices  about  training 
Contract  theory 
Attitude  -  Locus  of  control 
Making  training  seem  important 
Lead  by  example  -  attitude 
Mentors 

Groups  -  reward  structure 

Paired  post-training 

Teams  -  cooperative  controversy 
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PROBLEM  SOLVING  WITHOUT  EXAMPLES 


Description 

Oftentimes  when  we  are  asked  to  solve  problems  we  do  so  by 
modeling  our  solution  on  other  problems  for  which  we  already  have 
solutions.  This  may  make  things  easier  in  the  short  run,  but  it  may 
hinder  long-term  learning. 

How  to  do  it 

If  you  are  using  problem  solving  as  a  form  of  instruction,  then  what 
you  might  want  to  do  is  not  give  them  something  to  model  their 
answers  on.  If  they  have  a  similar  problem,  then  to  solve  the  new 
problem  the  learner  may  be  able  to  "proceed"  by  matching  the 
surface  features  of  the  problem  without  understanding  its  underlying 
structure. 

If  no  sample  upon  which  to  model  their  work  is  available,  the  learner 
has  no  alternative  but  to  figure  the  problem  out  from  principles. 
Assuming  those  are  what  you  want  the  person  to  learn,  this  is  a 
desirable  result. 

Some  caution  is  needed  to  make  the  level  of  challenge  appropriate.  If 
the  learner  is  unable  to  orient  himself  or  herself,  then 
discouragement  and  a  dislike  of  the  subject  are  likely  outcomes. 

Related  techniques 

Those  in  these  sections  and  the  sections  about  simple  memory  tricks. 
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STRESS  -  CRITERION  SITUATION  PRACTICE 


Description 

When  the  task  is  stressful,  it  is  often  not  feasible  to  practice  it  in  its 
full  form.  But  it  is  important  for  the  learner  to  encounter  and  learn  to 
deal  with  the  stress  that  he  or  she  will  encounter  in  the  actual  task. 

How  to  do  it 

Putting  people  in  what  are  called  criterion  situations  can  help.  A 
"criterion  situation"  in  training  reproduces  the  intensity  of  actual  task 
performance.  It  helps  with  generalization,  physiological 
defenses/toughness,  and  it  can  reduce  uncertainty  and  fear. 

Exposing  someone  to  stress  allows  them  to  become  used  to  it.  It  also 
means  that  they  will  form  a  realistic  idea  of  what  the  situation  is  like, 
which  will  probably  lower  the  amount  of  stress  they  feel  when 
encountering  it  (unless  it  is  really  more  dangerous  than  they  thought, 
in  which  case  they  probably  shouldn't  be  doing  it  anyway). 

Exposing  someone  to  stress  lets  them  learn  to  predict  what  will 
happen  next,  and  recognize  possible  problems/danger,  but  also 
when  there  is  nothing  to  worry  about. 

Letting  the  person  practice  doing  the  actual  task  in  the  criterion 
situation  is  important  because  it  allows  the  person  to  generalize  the 
learning  they  have  done  to  a  new  and  different  situation,  to  practice 
it  there,  and  to  dealing  with  the  stress  and  the  task  together. 

Unfortunately,  criterion  situation  training  can  also  hurt.  It  can 
interfere  with  task  acquisition  (learning),  instill  fears  and  sensitivity, 
lower  confidence,  create  despair,  and  cause  exhaustion. 

In  designing  and  administering  a  criterion  training  event,  the 
following  guidelines  can  be  used: 

1.  Trainees  should  be  allowed  to  familiarize  themselves  with  the 
stressors 

2.  Their  confidence  should  not  be  crushed  by  exposure  to  high 
stress  while  performing  task 

3.  Stress  should  not  be  allowed  to  hinder  acquisition  of  skills  or  to 
create  unrealistic  expectations  or  strategies 
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Indicators 

This  is  useful  when  the  task  is  stressful  and  when  the  causes  of  the 
stress  can  be  credibly  reproduced  in  a  training  environment. 

Counterindicators 

If  the  task  is  not  stressful,  do  not  use  this. 

Common  problems 

Stress  makes  people  narrow  their  focus,  and  in  doing  so  can  hinder 
practice  ability,  by  not  allowing  people  to  use  their  full  range  of 
cognitive  abilities,  or  concentrate  on  all  of  what  they  are  doing.  It 
also  makes  people  pay  attention  to  things  other  than  what  they  are 
being  trained  on.  This  means  less  ability  to  concentrate.  Both  of 
these  things  hinder  the  person's  ability  to  learn.  Learning  to  do  the 
task  away  from  the  stress  solves  these  problems  (to  a  good  extent). 

If  trainees  are  experiencing  too  much  stress,  give  them  more  time  to 
get  used  to  it  and  lower  the  difficulty  of  the  task.  Or,  if  trainees' 
confidence  seems  to  wane,  if  they  dread  training,  or  are  exhausted 
by  it,  easing  off  is  appropriate. 

Related  techniques 

Stress  -  Graduated  Training 

Stress  -  Phased  Training 

Stress  -  Emotional  Training 

Stress  -  Full  Information 

Automatization 

Overlearning 

Games  and  Simulations 

Expectations  Of  Training 

Train  With  A  Timer 

Don't  Take  Your  Time  -  Heuristics 

Sequence  -  Hard  Or  Easy 

Easy  First 
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EMOTIONAL  TRAINING  FOR  STRESS 

Description 

There  are  twp  ways  to  deal  with  stress.  Problem  solving  techniques 
are  ways  to  become  better  at  managing  the  environment  -  so  that  it 
is  less  stressful  and  so  that  you  feel  better  about  yourself.  Emotional 
techniques  are  used  to  manage  yourself  -  not  change  the 
environment,  but  to  relax,  let  yourself  focus,  etc. 

This  section  has  to  do  with  the  emotional  techniques.  There  is  a 
suggestion  about  how  to  best  make  use  of  problem  solving 
techniques,  but  no  specific  ways  of  changing  the  environment  to 
make  it  less  stressful. 

How  to  do  it 

Less  uncertainty  and  more  feeling  of  control  mean  less  emotional 
stress.  Stress  disrupts  normal  thinking  processes  -  it  makes  it  hard  to 
retrieve  things  from  memory  and  it  makes  it  hard  to  organize  and 
focus  your  thoughts.  (More  accurately,  it  is  easy  to  focus,  but  not  on 
the  right  things.)  A  three-phased  approach  can  be  used. 

Phase  One  is  to  convince  them  (or  confirm  for  them)  that  they  will  be 
able  to  control  their  reactions.  This  is  the  most  important  step. 

Phase  Two  is  to  train  them  in  skills  stress-management  skills  while 
not  under  stress.  For  example: 

•  Thought  restructuring,  where  negative  reactions  and  thoughts 
are  replaced  with  coping  thoughts  and  reactions,  or  attention  is 
consciously  redirected. 

•  Physiological  control,  such  as  breathing  and  muscle  relaxation 
exercises.  Reducing  the  physical  signs  of  stress,  such  as 
tenseness  and  heavy  breathing,  will  "trick"  you  into  thinking  that 
the  situation  is  less  stressful  than  it  seems  when  you  are 
breathing  hard. 

There  is  some  research  that  suggests  using  videotapes  of  people 
modeling  good  coping,  role  playing,  and  demonstrating  problems  that 
can  be  managed  with  these  techniques  all  help,  as  do  training 
sessions  of  at  least  one  hour  with  a  good  trainer.  (But  it's  hard  to  get 
too  excited  over  these  particular  findings,  given  the  measurement 
problems  and  the  interest  of  the  researches  to  find  what  they  were 
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looking  for.)  It  Is  important  to  provide  feedback,  so  that  the  trainee 
can  evaluate  his  or  her  progress. 

Phase  Three  is  to  let  them  practice  these  things  in  the  stressful 
situation,  or  alternately  in  a  graduated  introduction  of  the  task 
stressors.  As  the  trainees  habituate  to  the  situation,  they  will 
become  desensitized,  and  they  will  also  learn  to  recognize  what 
should  and  what  should  not  cause  alarm,  which  will  reduce  the  stress 
effect. 

Problem  solving  methods  are  preferable,  but  can  be  used  in 
combination  with  an  emotional  approach  if  the  level  of  stress  needs 
to  be  reduced  more  than  is  being  done  by  the  problem  solving 
approach  alone,  or  the  degree  of  efficacy  of  the  problem  solving 
methods  for  control  of  the  environment  is  limited. 

Indicators 

These  techniques  are  to  be  used  when  trainees'  ability  to  perform  is 
impaired  because  of  too  much  stress.  Emotional  techniques  are 
useful  when  the  problem  is  a  stressful  event  that  the  person  can  do 
nothing  about,  i.e.,  they  cannot  change  the  environment.  Emotional 
training  is  especially  helpful  for  events  that  are  brief  in  duration. 

Counterindicators 

If  the  stressor  is  something  that  the  person  can  change,  emotional 
techniques  may  be  still  useful  (depending  on  the  stress  level),  but 
should  be  used  in  support  of  (and  clearly  as  an  adjunct  to)  behaviors 
that  change  the  environment. 

Emotional  training  helps  with  stress  that  causes  anxiety  or 
depression,  but  not  with  things  like  sleep  deprivation,  noise,  heat, 
cold  or  altitude. 

Common  problems 

Over-desensitization  is  possible,  where  the  energy  that  the  stress 
stimulates  is  diluted.  This  can  result  in  lower  levels  of  concentration 
and  commitment  to  task  completion. 

It  is  also  true  that  some  kinds  of  emotional  stress  cannot  be 
simulated  -  for  example,  all  heart  surgeons  "lose"  patients  (a 
euphemism  for  having  the  people  they're  caring  for  die).  In  those 
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cases,  socialization  and  team-oriented  support  is  appropriate. 

Related  techniques 

Stress  -  Full  information 
Stress  -  Graduated  training 
Stress  -  Phased  training 
Stress  -  Criterion  practice 
Socialization 
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STRESS  -  FULL  INFORMATION 

Description 

Giving  people  full  information  about  what  they  may  encounter  can 
lead  to  problems  when  it  is  actually  encountered.  Arousal  can  be 
heightened  to  a  damaging  degree,  overly-aggressive  reactions  can 
be  made  more  likely,  and  so  on.  However,  it  can  also  be  harmful  to 
tell  they  don't  need  to  worry  if  that  misrepresents  the  actual  task 
situation.  Ideally,  learners  should  be  given  the  optimal  amount  of 
preparatory  (warning)  information  when  training  for  very  stressful 
tasks. 

How  to  do  it 

When  people  know  what  to  expect,  they  can  make  themselves  more 
ready  for  it,  increase  their  alertness,  and  be  primed  to  respond 
effectively.  Realistic  expectations  attenuate  stress  and  improve 
performance.  Therefore,  this  approach  advocates  telling  trainees 
what  might  happen,  in  conjunction  with  their  performance  training. 
Combining  appropriate  expectations  with  developing  task 
competence  gives  trainees  a  sense  of  control. 

There  are  three  steps. 

1.  Tell  them  they  can  do  it,  reassure  them,  tell  them  (or  better, 
show  them)  others  have  done  it  before  them. 

2.  Tell  them  when  it  is  going  to  start,  and  also  when  it  is  going  to 
stop.  Telling  people  how  many  missions  they  have  to  fly  reduces 
stress,  and  telling  them  how  much  further  they  have  to  march 
not  only  helps  mentally,  but  in  one  study  it  actually  changed  the 
composition  of  blood  samples  taken  partway  through. 

3.  Tell  them  how  to  monitor  the  situation  and  provide  them  with 
feedback-rich  practice  opportunities,  so  they  can  confidently  tell 
when  things  are  going  well,  and  when  the  danger/pressure  is 
and  is  not  present. 

The  types  of  information  that  can  be  provided  include: 

1.  Sensory  information,  so  that  they  can  appropriately  interpret 
what  they  feel,  see,  hear,  etc.,  without  panic 

2.  Procedural  Information,  that  is,  what  to  expect,  what  sequences 
are  typical,  and  common  errors  to  avoid. 

3.  Instrumental  information,  so  that  they  know  specific  techniques 
to  cope  with  the  stressors  and  their  effects.  Having  such 
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techniques  gives  trainees  both  actual  and  perceived  control  over 
the  task  situation. 

If  there  is  information  that  will  neither  make  trainees  do  better  or  feel 
better,  there  is  no  reason  to  provide  it,  especially  if  it  is  likely  to 
heighten  stress  when  the  event  occurs. 

But  trainees  should  not  be  told  there  is  nothing  to  worry  about  if 
there  is.  Unreliable  (false)  information  is  bad,  even  if  it  is  temporarily 
reassuring,  because  its  invalidation  when  events  actually  happen  can 
make  things  worse  by  casting  doubt  on  the  reliability  of  other 
elements  of  their  training. 

Indicators 

The  suggestions  above  are  for  tasks  that  involve  (at  least  potentially) 
very  high  levels  of  stress  -  where  there  may  be  fear  and  danger.  The 
method  is  also  applicable  where  trainees  may  be  made  very 
uncomfortable  by  the  task  situation. 

In  these  situations,  the  amount  of  information  trainees  have  about 
what  is  happening  and  what  is  going  to  happen  is  very  important  to 
maintenance  of  a  positive  attitude. 

The  more  useful  a  piece  of  information  is  to  the  person,  the  worse  an 
idea  it  is  to  withhold  it,  even  if  providing  the  information  will  heighten 
stress. 

Counterindicators 

If  reactions  to  the  particular  stressors  of  the  task  domain  are  highly 
variable  -  for  example,  some  people  find  public  speaking  so  anxiety- 
provoking  as  to  lead  to  nausea,  while  others  relish  it  -  then  building 
expectations  of  stress  may  themselves  introduce  stress. 

Common  problems 

If  trainees  develop  fears  of  training  before  they  start  practicing  (after 
only  being  told  about  it),  or  if  they  overreact  to  certain  stressors, 
they  may  be  being  too  much. 

If  people  are  told  what  to  worry  about  without  being  told  what  to  do 
about  it,  stress  that  could  be  avoided  without  withholding  information 
is  probably  being  created. 
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If  a  trainee  does  something  and  his  or  her  fears  are  not  realized,  he 
or  she  gains  self-efficacy.  However,  if  the  trainee  does  something  and 
gets  scared  or  overly  upset,  then  even  if  he  or  she  accomplishes  the 
task,  he  or  she  will  still  lose  self-efficacy.  It  is  the  trainer's 
responsibility  to  create  an  environment  and  an  atmosphere  where 
trainees  can  practice  the  task  under  stress  without  these  conditions 
becoming  overwhelming. 

Related  techniques 

Stress  -  Criterion  Practice 
Stress  -  Emotional  Training 
Stress  -  Graduated  Training 
Stress  -  Phased  Training 
Enactive  Mastery 
Limit  Introduction 
Don't  Take  Your  Time  -  Heuristics 
Attitude  -  Locus  Of  Control 
Feedback  -  Delay  It 
Feedback  -  Withhold  It 

References 

Keinan  and  Friedland 
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STRESS  -  GRADUATED  TRAINING 

Description 

Sometimes  a  task  is  too  stressful  for  immediate,  immersive  practice. 
The  technique  here  is  to  start  with  mild  levels  of  stress,  and  gradually 
increase  the  realism  until  the  the  desired  level  ("criterion  level")  is 
reached. 

How  to  do  it 

In  graduated  training,  stressors  are  introduced  into  the  practice 
environment  according  to  these  two  guidelines: 

1.  Do  not  proceed  to  the  next  level  until  an  acceptable  degree  of 
proficiency  is  reached.  Otherwise  the  additional  stress  will  simply 
make  it  harder  for  learning  to  be  achieved.  It  may  actually  cause 
the  trainees  to  learn  things  that  are  not  true,  and  thereby  get 
worse,  not  better.  (For  the  problems  this  can  cause,  see 
Automaticity  and  Overlearning). 

2.  Tell  trainees  what  levels  of  stress  they  can  expect  to  reach,  or 
they  may  fall  into  despair.  One  study  found  that  when  trainees 
are  not  told  they  have  reached  the  limit,  they  do  less  well  than 
when  they  are.  Trainees  may  loose  confidence  or  give  up  if  they 
think  they  will  not  be  able  to  make  it.  Conversely,  it  is  reassuring 
and  rewarding  to  think  you  will. 

Indicators 

This  method  can  be  used  when  there  are  high  levels  of  stress 
requiring  gradual  adaptation. 

Counterindicators 

If  stressors  come  in  discrete  "packets,"  such  that  they  cannot  be 
decomposed  into  elements  that  can  be  introduced  gradually,  this 
method  cannot  be  applied. 

Common  problems 

Trainees'  self-reports  on  adaptation  may  not  be  reliable,  so 
performance  should  be  carefully  monitored  and  impatience  avoided. 

Related  techniques 

Stress  -  Criterion  Practice 
Stress  -  Phased  Training 
Stress  -  Emotional  Training 
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Automatization 

Overlearning 

Don't  Take  Your  Time  -  Heuristics 
Sequence  -  Hard  Or  Easy 
Easy  First 

References 

Keinan  and  Friedland 
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STRESS  -  PHASED  TRAINING 

Description 

Instead  of  placing  a  trainee  directly  into  a  training  situation  that  is 
highly  stressful,  this  method  suggests  three  distinct  phases,  intended 
to  allow  the  trainee  to  become  accustomed  to  the  stress  and  also  to 
learn  the  skills  well. 

How  to  do  it 

Keinan  and  Friedland  propose  three  requirements  for  stress  training: 

1.  Trainees  should  be  allowed  to  familiarize  themselves  with  the 
stressors. 

2.  Their  confidence  should  not  be  crushed  by  exposure  to  high 
stress  while  performing  task. 

3.  Stress  should  not  be  allowed  to  hinder  acquisition  of  skills  or  to 
create  unrealistic  expectations  or  strategies. 

Following  this  philosophy,  they  recommend  a  three-phased  training 
approach. 

1.  In  Phase  1,  train  the  subject  to  do  the  task  under  minimal  stress. 

2.  In  Phase  2,  expose  the  subject  to  the  stressors  they  can  expect 
without  asking  them  to  do  anything. 

3.  In  Phase  3,  have  the  trainees  perform  the  task  under  the 
criterion  level  of  the  stressors. 

However,  in  experiments  by  the  authors,  the  finding  emerged  that 
just  steps  1  and  3  together  worked  as  well  as  steps  1,  2,  and  3. 
Neither  1  and  2  only,  1  only,  nor  3  only  were  as  effective. 

Indicators 

This  technique  can  be  effective  if  the  stressors  can  be  separated  and 
manipulated  independently  from  the  skills  of  performance. 

Counterindicators 

If  dealing  with  the  stressors  is  intrinsic  to  the  task  skill,  this  technique 
is  inapplicable. 

Common  problems 

The  issues  for  this  method  are  rate  of  introduction  of  stressors  and 
stressor  realism.  Both  need  to  be  carefully  managed  to  achieve 
effective  transfer. 
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Related  techniques 

Stress  -  Graduated  Training 
Stress  -  Criterion  Practice 
Stress  -  Emotional  Training 
Automatization 
Overlearning 

Don't  Take  Your  Time  -  Heuristics 
Sequence  -  Hard  Or  Easy 
Easy  First 

References 

Keinan  and  Friedland 
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SUMMARIZE  FOR  THE  CLASS 

Description 

Oftentimes  it  is  easy  to  go  through  a  lecture  without  really  putting 
much  thought  into  what  is  being  said.  This  technique  is  a  way  to 
make  students  responsible  for  knowing  what  was  said,  and  for 
understanding  what  the  material  really  means. 

How  to  do  it 

The  method  has  students  read  (or  watch  a  lecture  or  movie  or 
whatever),  then  summarize  the  content  for  the  class.  The  student 
should  be  encouraged  to  ask  questions  that  get  at  the  heart  of  what 
they  have  just  learned,  and  in  turn  to  answer  questions  when  they 
are  summerizing. 

It  can  be  useful  for  the  teacher  model  this  behavior  (by  summarizing 
and  asking  questions)  before  the  students  are  asked  to  do  it.  When 
they  begin,  it  may  be  supportive  to  be  prompted  by  the  teacher  at 
first.  As  they  develop  the  necessary  skills  they  can  summarize  on 
their  own. 

A  useful  variation  is  to  have  the  students  summarize  and  formulate 
questions  on  their  own. 

When  the  learner  knows  that  he  or  she  is  going  to  have  to  come  up 
with  the  important  questions  at  the  end  of  the  day,  he  or  she  is  likely 
to: 

•  Think  about  the  subject  matter  with  the  goal  of  understanding  it 
(not  just  listening  to  it,  as  we  often  do,  I  think). 

•  Think  about  what  the  most  important  things  to  come  away 
understanding  are  (so  he  or  she  can  ask  questions  about  them). 

•  Think  about  what  parts  of  it  make  sense  with  respect  to  other 
things  already  known  (the  ones  that  don't  the  learner  is  highly 
motivated  to  question). 

•  Know  the  answers  to  his  or  her  own  questions. 

This  approach  is  appropriate  for  instructional  settings  -  lecture, 
movie,  classroom,  reading  -  where  the  learner  doesn't  either  need  or 
get  to  do  a  lot  of  practicing. 

If  the  subject  matter  is  not  complex,  and  does  not  have  some  sort  of 
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interesting  structure  intrinsic  to  it  (e.g.,  memorizing  zip  codes),  this 
method  is  inappropriate.  It  is  unlikely  to  be  applicable  to  procedural 
knowledge. 

Example 

"I  sit  through  class  everyday,  bored.  Then  one  day  the  teacher  points 
at  me  and  says  'You  are  going  to  lead  the  class  in  a  discussion  by 
asking  them  questions  at  the  end  of  today's  class.' 

"So  I  open  up  my  notebook,  and  start  taking  notes.  I  think  about 
what  the  teacher  says,  and  each  thing  that  comes  up  I  say  'what  do  I 
think  of  that?  Do  I  believe  it?  Do  I  know  something  else  about  it  that 
is  interesting?  Does  it  make  sense  with  the  other  things  that  I  have 
learned  about  today  in  class?  Or  on  other  days  in  class?  Is  there 
anything  about  it  that  is  worth  discussing,  or  that  I  doubt  or  don't 
understand?  And  is  this  the  answer  to  that  other  question  that  I  had 
before?'  All  of  these  things  make  me  really  think  about  the  lecture  as 
I  go  through  it,  and  not  only  think  about  what  is  being  said,  but 
about  its  importance,  why  it  matters,  and  other  things  I  know. 

"At  the  end  of  class,  I  have  a  much  better  understanding  of  what  was 
said  than  I  did  in  other  classes  becuase  I  have  participated  in  my 
own  learning. 

"I  then  pick  the  things  that  I  think  are  the  most  important,  out  of  the 
questions  that  I  have  thought  of.  And  I  bring  them  up.  When  I  do, 
the  class  response  lets  me  know  what  I  missed  and  what  I  am 
unable  to  explain,  and  of  course  what  I  got  right." 

Related  techniques 

Those  in  these  sections  and  the  sections  about  simple  memory  tricks. 

References 

Glasser  and  Bassok 
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TEAMS:  CROSS  TRAINING 

Description 

In  cross  training  team  members  train  a  little  bit  in  other  people's 
positions.  By  learning  what  other  team  members  have  to  do  during  a 
task,  they  can  anticipate  their  needs,  and  understand  an  overall 
framework.  In  this  way,  team  members  become  more  efficient,  and 
the  whole  team  learns  to  coordinate  without  unnecessary 
communication. 

How  to  do  it 

Have  team  members  rotate  from  position  to  position  or  from  role  to 
role  during  practice  sessions.  Limit  the  number  of  people  switching  at 
any  one  time  or  performance  levels  are  likely  to  drop  to  a  point  where 
the  task  falls  apart. 

Rotation  can  occur  with  the  whole  task  or  on  simplified  versions. 

When  job  rotation  is  not  feasible  because  of  safety,  for  example, 
provide  controlled  simulations  instead  of  the  real  thing.  If  a  position 
or  role  involves  complicated  actions,  it's  probably  wise  to  allow  a 
short  period  of  passive  observation  before  the  trainee  switches  to  it. 

Role  or  position  switching,  timed  well,  and  probably  without  the 
whole  team,  may  also  be  useful  to  resolve  persistent  mistakes 
without  excessive  criticism:  A  trainee  can  take  the  role  where  he 
experiences  the  effect  of  something  he  has  been  doing  incorrectly. 
This  may  be  enough  to  reveal,  for  example,  a  misunderstood  priority 
or  inaccurate  procedure. 

Allow  time  for  feedback  and  watch  for  improved  efficiency  in  both 
original  roles  and  overall  team  performance.  Ask  for  accurate 
explanations  from  team  members  about  their  teammates'  behavior, 
and  for  descriptions  of  their  needs.  Trainees'  understanding  should 
increase  with  respect  to  why  other  team  members  do  what  they  do, 
what  difficulties  are  involved,  and  how  their  own  actions  effect  those 
of  others.  Interdependence  and  mutual  responsibility  will  become 
clearer. 

Indicators 

You  may  want  to  use  cross  training  if: 
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•  Interdependence  is  important. 

•  Team  members  need  to  understand  what  the  others  are  doing 
and  why. 

•  Team  members  need  to  anticipate  each  others  actions. 

•  Team  members  need  greater  understanding  of  teammates' 
needs. 

The  more  different  the  different  jobs  or  roles  are,  the  more  useful 
cross  training  is. 

Counterindicators 

Cross  training  can  be  wasteful  if  time  is  limited  and  would  be  better 
devoted  to  each  trainee  learning  his  or  her  own  part.  Some  tasks 
may  be  too  simple  to  provide  any  benefits  from  cross  training.  Others 
may  involve  highly  specialized  roles  which  make  cross  training  either 
unnecessary  or  impossible.  In  some  group  training,  team  members 
may  work  together,  but  without  relying  on  each  other  for  task 
performance. 

Common  problems 

Original  team  members  for  any  role  or  position  may  be  too  eager  to 
correct  trainees  new  to  the  role  or  position.  Make  sure  all  members 
are  aware  that  time  will  be  set  aside  for  commentary  and  feedback. 
Some  team  members  may  feel  work  on  roles  beside  their  own  is  a 
waste  of  time.  Use  socialization  and  a  positive  environment  to  get 
over  this.  Cooperative  controversy  may  also  be  useful  here. 

Related  techniques 

Teams  -  cooperative  controversy 
Enactive  mastery 
Contract  theory 
r  Behavioral  modeling 

References 
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TEAMS  -  GENERAL  METHODS 

Description 

When  people  have  to  work  as  teams,  how  team  members  interact  is 
important.  It  can  be  helpful  to  train  the  team  in  teamwork  skills. 

How  to  do  it 

The  more  people  need  to  coordinate  and  the  more  they  rely  on  each 
other,  the  more  benefit  they  will  derive  from  team  tasks.  The  need  for 
teamwork  exists  when: 

•  There  is  a  great  deal  of  interdependence. 

•  Environmental  stability  is  low.  If  a  task  involves  a  lot  of  variability 
and/or  unpredictability,  the  team  is  going  to  benefit  more  from 
being  able  to  work  as  a  team  -  monitoring  each  other,  predicting 
responses,  working  toward  the  same  goal,  anticipation,  all  of 
which  are  more  important  with  less  environmental  stability. 

•  There  is  low  team  member  turnover. 

•  There  is  low  membership  in  multiple  teams. 

•  There  are  many  different  tasks  the  team  must  perform. 

The  more  these  things  are  true,  the  more  important  it  is  to  train  the 
members  to  work  as  a  team. 

When  a  team  has  to  function  under  stress,  or  in  trying  circumstances, 
it  is  especially  important  for  them  them  to  be  able  to  rely  on  each 
other,  have  a  common  goal,  anticipate  each  other,  and  so  on:  It  is 
especially  important  that  they  work  as  a  team. 

Teamwork  skills  training  should: 

1.  Focus  on  teamwork  skills,  such  as  communication,  compensatory 
behavior,  leadership,  assertiveness,  decision  making,  planning, 
situational  awareness. 

2.  Provide  information  on  teamwork,  demonstrations  of  it,  practice 
(role  playing  and  simulation)  of  teamwork,  and  feedback  on 
teamwork  quality. 

3.  Provide  for  measurement  and  diagnosis  of  skills  from  moment  to 
moment. 

4.  Provide  both  experience  and  confidence  in  working  as  a  team 
when  faced  with  a  given  task. 

5.  Provide  training  in  "generic"  teamwork  skills  such  as 
communication  and  leadership,  and  interpersonal  skills. 
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When  small  groups  of  people  work  together  towards  a  common  goal, 
possibly  because  they  are  relying  on  each  other  and/or  because  they 
are  doing  things  for  one  another,  they  tend  to  form  a  bond.  This  can 
enhance  the  teamwork  effect. 

Common  problems 

Egotistical  team  members  can  be  an  issue.  Firm  leadership  can 
ameliorate  this  to  some  extent. 

Related  techniques 

Teams  -  Cross  training 
Cooperative  learning 

References 
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TEAMS:  COOPERATIVE  CONTROVERSY 

Description 

Cooperative  controversy  trains  team  members  to  maximize  the 
effectiveness  of  cooperation.  It  does  this  by  encouraging  constructive 
opposition  and  exploration  of  new  ideas.  This  occurs  within  a 
framework  of  established  norms  for  friendly,  honest  exchange  and 
feedback. 

How  to  do  it 

This  approach  is  based  on  several  premises  Which  ideally  should  be 
integral  to  all  training.  Establish  that  everyone  is  "in  this  together." 
Insults  and  personal  attack  are  unacceptable  -  ideas,  not  people  are 
open  to  challenge.  Force  and  dominance  are  not  permitted, 
persuasion  and  respect  are.  Questions  are  welcome  as  well  as  other 
techniques  to  clarify  information  and  to  understand  other  people's 
perspectives.  The  golden  rule  for  everyone  involved:  Bring  things  up 
with  others  as  you  would  want  them  to  do  with  you;  listen  to  them, 
as  you  would  want  them  to  listen  to  you. 

Explain  that  you  want  to  step  up  the  level  of  cooperation  by  using  it 
as  a  platform  to  challenge  accepted  norms.  Cooperation  for 
cooperation's  sake  is  not  a  good  idea  if  it  leads  only  to  a  plateau  of 
progress  or  reinforces  wrong  practices.  Essential  to  cooperative 
controversy  is  the  willingness  of  everyone  not  only  to  influence,  but 
also  to  be  influenced.  The  result  you  are  looking  for  is  mutually 
advantageous  solutions. 

One  of  the  most  basic  uses  of  this  technique  is  to  set  aside  a  time 
before  or  after  training  sessions  to  discuss  views,  ideas,  opinions  and 
frustrations.  Providing  a  formal  basis  for  cooperative  controversy 
avoids  explosive  and/or  simmering  disagreements  during  training.  It 
also  builds  patterns  of  constructive  engagement  when  time-outs  are 
needed  to  resolve  conflict.  Encourage  trainees  to  argue  their  beliefs 
during  discussion  and  see  that  odd  ideas  are  not  simply  dismissed. 
There  should  be  no  retribution  for  speaking  out  and  therefore  no  fear 
of  doing  so. 

Set  up  cooperative  controversy  more  formally  by  assigning  opposing 
views.  This  will  get  a  dialog  going  and  serves  as  practice  for  self- 
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initiated  exchange.  In  addition,  one  person  at  a  time  can  be  assigned 
the  role  of  criticizing  the  training  project,  helping  to  detect  problems. 

You  can  take' the  technique  one  step  further.  Ask  trainees  to  prepare 
ahead  of  time  by  consulting  various  sources.  They  should  seek  out 
views,  printed  or  otherwise,  that  disagree  with  each  other  and  with 
you.  When  these  are  raised  in  group  discussion,  remind  trainees  that 
the  aim  is  to  use  the  material  to  create  useful  solutions. 

Indicators 

Cooperative  controversy  is  useful  for  long  term  teamwork.  It  can  also 
be  used  in  situations  where  you  think  people  are  holding  back  or 
when  they  are  in  continuing  dispute  about  training  matters.  It  may 
also  help  trainees  who  feel  offended  when  they  are  criticized. 

Counterindicators 

If  training  is  very  short,  urgent  or  highly  customized,  cooperative 
controversy  will  not  be  very  useful.  Very  large  groups  may  need 
subdividing  if  this  technique  is  to  be  effective. 

Common  problems 

Good  discipline  is  essential  to  ensure  that  cooperative  controversy 
does  not  turn  into  a  free-for-all.  Human  nature  can  lead  discussion 
into  personal  issues  and  away  from  a  basic  exchange  of  ideas.  In  this 
way,  cooperative  controversy  can  spill  out  of  its  allotted  time  and 
result  in  the  sacrifice  of  other  effective  training  methods.  Strong, 
diplomatic  leadership  is  needed  to  keep  this  technique  on  track. 

Related  techniques 

Behavior  Modeling 
Establish  An  Environment 
Games  And  Simulations 
Paired  Post-Training 
Sequence  -  Hard  Or  Easy 
Stress  -  Emotional  Training 
Summarize  For  Class 
Teams  -  General  Methods 
Teams  -  Cross  Training 
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TIMING  OF  PRACTICE 


Description 

Timing  of  practice  has  three  parts:  Timing  practice  so  as  to  get  the 
most  out  of  if;  breaking  practice  into  appropriately-sized  sessions; 
and  if  needed  to  effectively  perform  the  task,  developing  temporal 
independence,  the  ability  to  do  the  work  regardless  of  time  of  day. 

The  gist  is  to  break  practice  into  absorbable  chunks,  space  practice 
so  it's  best  received,  and  vary  practice  time  to  generalize  the  skill. 

How  to  do  it 

Two  straightforward  observations  influence  the  timing  of  practice: 

•  People  practice  better  when  they  are  not  fatigued. 

•  People  absorb,  integrate,  and  memorize  unconsciously  when  not 
practicing. 

Taken  together,  these  two  facts  suggest  that  practicing  in 
"absorbable",  appropriately-timed  sessions  will  be  more  effective 
than  cramming.  If  fourteen  hours  are  allotted  for  training  a  particular 
task,  the  scheduling  seven  days  of  two  hour  sessions,  timed  for 
when  trainees  are  alert  and  ready,  will  result  in  better  learning  and 
retention  than  a  single-day  fourteen  hour  intensive. 

The  special  case  is  with  tasks  where  an  important  component  is 
being  able  to  perform  effectively  when  fatigued.  Firefighting  is  an 
example.  Not  only  must  firefighters  perform  under  taxing  conditions, 
but  they  must  monitor  their  performance  to  determine  when  an 
unacceptable  (potentially  dangerous)  level  of  fatigue  has  been 
reached.  In  this  kind  of  case,  timing  of  practice  would  be  explicitly 
directed  at  inducing  fatigue. 

For  some  tasks— firefighting  is  an  example,  as  are  police  work  and 
medicine— effective  practitioners  must  be  able  to  perform  well  no 
matter  what  time  of  day  or  night  they  are  called  upon.  "Temporal 
relocation"— the  ability  to  perform  no  matter  when  called  upon— like 
any  skill  is  developed  through  practice.  Timing  practice  to  give  the 
trainee  opportunity  to  perform  the  task  in  a  wide  range  of  temporal 
settings  improves  the  ability  to  generalize. 

Time  not  thinking  about  something  may  actually  be  necessary  to 
learn  it.  Recent  work  reports  a  roughly  six-to-eight  hour  consolidation 
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period  for  new  (physical)  knowledge  to  transfer  from  short-  to  long¬ 
term  memory.  The  study  team  found  that  overlaying  other  new  and 
related  skills  on  the  target  learning  during  this  consolidation  period 
resulted  in  diffusion  or  even  elimination  of  memory  transfer. 

Students  and  trainees  commited  to  mastering  a  skill  know  that 
working  hard  works,  but  Ericsson  has  found  in  his  research  on 
expertise  that  working  beyond  cconcentration  is  actually  harmful.  For 
self-directed  practice,  the  drive  to  work  hard  has  to  be  balanced  with 
the  need  to  take  breaks— not  so  many  breaks  as  to  lose  the  thread, 
but  then,  being  overly  dogged  is  ineffective  and  may  be  harmful  as 
well.  What  to  do? 

This  is  really  a  number  of  questions  disguised  as  one.  The  right 
approach  will  depend  on  the  nature  of  the  task,  the  number  of 
competing  taks  that  must  be  performed  or  learned,  variations  in 
student  interest  in  the  task,  and  cyclical  variations  in  the  ability  to 
concentrate. 

A  common  thread  through  all  of  these,  though,  is  self-management.  If 
in  conjunction  with  the  timing  of  training,  a  specific  goal  is  set  for  the 
learner  to  develop  his  or  her  study  timing  skills,  then  both  can  be 
learned  concurrently.  And  it  is  easy  to  argue  that  the  self¬ 
management  skills  will  continue  to  grow  in  value  as  time  goes  on. 

If  practice  is  trainer-controlled,  rather  than  self-directed,  it  is  the 
trainer's  responsibility  to  plan  the  practice  sessions  realistically  with 
respect  to  the  current  capacities  of  his  or  her  trainees.  Finding  the 
right  level  may  be  a  process  of  trial-and-error,  but  close  observation 
of  trainees  performance  and  attitudes  can  hasten  this  process. 

It  is  less  of  special  case  than  it  might  seem  when  training  a  skill  that 
must  become  automatic  (see  Section  XXX).  In  this  case,  though,  once 
the  basic  skill  is  mastered  the  student's  concentration  still  needs  to 
be  available,  if  only  to  be  diverted  so  that  practice  of  the  automatic 
skill  can  continue  while  the  learner's  conscious  attention  is  directed 
elsewhere. 

A  benefit  of  well-timed  breaks,  whether  self-initiated  or  specified  by 
the  trainner,  is  that  practice  occurs  at  different  times.  In  this  way, 
learning  is  generalized  to  different  temporal  settings.  A  caution  on 
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this— if  it  is  desirable  for  the  time  or  the  place  to  be  a  cue  to  initiate 
the  target  behavior,  then  care  in  varying  either  is  called  for. 

Finally,  a  rule  of  shaping  can  be  used  to  help  time  breaks:  Always 
stop  at  a  high  point. 

Indicators 

Whenever  developing  skill  in  a  task  requires  hours  of  practice  or 
study  a  day  (or  even  hours  a  week),  timing  of  practice  sessions,  both 
length  and  time  of  day,  should  be  consciously  considered.  This  is 
especially  true  if  the  trainees  or  students  set  their  own 
training/study/practice  schedules.  To  the  extent  that  flexibility  to 
adjust  is  present,  timing  of  practice  is  something  to  plan  with  some 
thought. 

Counterindicators 

Sometimes  it  is  impractical  to  space  practice.  Access  to  equipment, 
materials,  or  meeting  space,  the  logistics  of  gathering  people 
together,  restrictions  on  the  instructor's  schedule— any  of  these  can 
constrict  the  choices  of  timing. 

With  some  skills  and  knowledge,  prolonged  practice  is  effective.  For 
example,  when  becoming  acquainted  with  new  knowedge,  immersion 
may  be  more  effective  than  taking  breaks  of  any  length  long  enough 
to  let  the  various  new  elements  slip  from  the  mind.  This  seems 
especially  true  when  the  material  is  new  enough  that  significant 
mental  energy  must  go  into  becoming  oriented,  building  mental  maps 
of  how  the  various  newly-encountered  pieces  all  fit  together.  Note, 
however,  that  this  is  consistent  with  the  notion  of  not  exceeding  the 
range  of  concentration.  It  may  be  the  case  that  an  implicit  goal  of 
practice  is  to  expand  that  range. 

Common  problems 

It  is  satisfying  for  trainees  to  feel  like  they  have  worked  hard,  and  for 
motivated  trainees  it  is  easy  to  feel  guilty  about  not  working  enough. 
However,  sometimes  working  more  now  means  being  much  less  able 
to  work  later.  Letting  trainees  push  themselves  beyond  their  ability 
to  absorb  and  integrate  the  material  they  are  trying  to  learn  not  only 
does  no  good,  it  may  do  harm.  It  is  the  trainers  responsibility  to 
monitor  this  and  be  certain  it  doesn't  happen. 

Trainees  should  be  taught  to  monitor  their  own  concentration.  When 
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they  work  past  their  ability  to  concentrate,  stopping  is  almost  always 
better  than  developing  a  habit  of  performing  the  task  with 
wanndering  concentration  or  unconsciously  learning  erroneous 
performance.  Returning  to  the  work  later  will  provide  greater  benefits 
than  dogged  .but  diffuse  pursuit.  For  some  tasks,  even  a  five-minute 
break  can  be  beneficial. 

Related  techniques  for  this  training  problem 

Covert  practice 
Deliberate  practice 
Automaticity 
Immediate  practice 

Practice  alone  (space  practice  when  alone) 

Stress  -  How  to  practice 

References 
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TRANSFER  -  IMMEDIATE  APPLICATION 


Description 

Pentland  (1989)  found  that  having  trainees  apply  their  new  skills 
immediately  upon  returning  to  their  jobs  resulted  in  a  major 
improvement  in  long  term  retention.  This  means  training  should  be 
timed  so  that  the  learned  skills  can  be  immediately  applied,  and  that 
the  training  should  be  coordinated  with  the  trainees'  management  so 
that  assignments  are  made  such  that  the  trainees'  new  skills  are 
actually  used. 

Immediate  application  aids  generalization:  The  trainee  will  need  to 
learn  to  do  the  task  in  the  real  situation  and  environment.  No  matter 
how  realistic  training  is,  it  is  still  training. 

"Generalization"  is  an  important  aspect  of  any  training.  Tasks  are 
associated  with  environments,  with  where  and  when  you  do  them. 
The  more  strongly  the  task  is  restricted  to  a  single  environment  (a 
surgical  theater,  a  laboratory),  the  stronger  the  association  of  task 
with  environment.  The  positive  benefit  of  this  is  that  the  environment 
comes  to  aid  task  performance,  by  providing  a  set  of  cues  that  trigger 
appropriate  responses  in  the  performer.  A  strong  association  with  a 
single  environment  is  negative,  however,  if  the  task  needs  to  be 
performed  in  a  variety  of  environments,  whether  that  variation  is 
physical,  social,  temporal,  or  all  three.  If  that's  the  case— it  usually 
is— task  performance  needs  to  be  generalized. 

Immediate  practice  on  the  job  is  always  beneficial  to  generalization 
because  at  the  minimum  it  helps  generalize  from  the  training  to  the 
real-world  environments.  It  is  usually  beneficial  because  the  job  will 
probably  have  more  environmental  variation  than  the  training.  For 
the  trainee  to  integrate  his  or  her  new  skills  into  the  total  fabric  of 
the  work,  that  is,  to  generalize  them,  real-world  application  is 
necessary. 

On  the  other  hand,  if  you  train  people  but  they  have  no  way  to  use 
the  training  when  they  return  to  their  work,  your  former  trainees  will 
probably  make  a  couple  of  simple  inferences:  "That  training  was  a 
waste  of  time;"  and,  "the  people  who  designed  that  training  don't 
have  a  clue  about  what  we  really  do."  This  may  not  be  the  desired 
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outcome  of  your  training  plan. 

How  to  do  it 

Make  certain  trainees  begin  performing  the  task  with  adequate 
regularity  immediately  after  they  finish  training. 

Doing  it  right  off  the  bat  makes  it  into  something  they  will 
subsequently  expect  to  do.  This  in  turn  cements  the  importance  of 
the  new  skill,  and  of  its  continuing  to  development.  On  the  other 
hand,  if  on  return  to  work  chances  for  application  of  the  skill  just 
don't  arise,  people  will  come  to  believe  that  the  new  skills  may  be 
irrelevant,  not  important  enough  to  devote  any  mental  energy  to 
remembering.  The  longer  they  go  before  they  can  apply  their  new 
skills,  the  greater  the  potential  decay  of  training  effect.  This  can  even 
be  exacerbated  by  a  potential  increase  in  any  fear  or  sense  of 
intimidation  they  may  have  (or  may  develop). 

Indicators 

The  more  complex  the  skill,  or  the  more  complex  the  task 
environment  into  which  the  skill  is  to  be  integrated,  the  more 
important  immediate  practice  is.  If  there  is  a  high  proportion  or 
procedural  knowledge,  this  also  increases  the  importance  of 
immediate  application. 

Counterindicators 

Immediate  practice  may  not  be  possible.  If  you  train  firefighters  in 
chemical  spill  containment  procedures,  it  is  probably  not  a  good  idea 
to  derail  a  train  so  they  can  get  real-world  practice.  (This  is  not  a 
book  about  law,  though.  State  regulations  may  vary!)  If  immediate 
practice  is  impossible,  very  difficult,  or  simply  inappropriate,  then 
periodic  refresher  training  and/or  practicing  parts  of  the  task  may 
help. 

Common  problems 

If  for  some  reason,  training  can't  be  optimally  timed,  this  should  be 
explained  to  the  trainees  so  that  they  have  appropriate 
expectations.  For  instance,  perhaps  a  kind  of  heavy  equipment  is 
being  trained.  It  may  be  that  the  company  isn't  due  to  receive  the 
equipment  for  six  months,  but  now  is  the  only  time  the  trainers  are 
available.  If  situations  like  this  are  unavoidable,  then  full  disclosure  to 
the  trainees  should  be  made. 
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When  they  go  back  to  the  job  and  begin  applying  their  new 
knowledge,  it's  beneficial  if  they  apply  all  of  what  they  have  been 
taught.  Commonly,  people  remember  only  part  of  what  they  have 
been  taught.  Then  when  they  practice,  only  that  part  gets  cemented. 
If  the  person  comes  to  believes  they  have  learned  the  skill,  or 
learned  enough  of  the  skill  to  get  by,  the  material  that  was 
inadequately  practiced  will  be  forgotten  unless  the  task  environment 
absolutely  demands  it.  (People  are  remarkably  clever  about  getting 
by  with  whatever  skill  set  they  have.)  Unobstrusive  observation,  or 
delayed  traininng  refreshers,  can  help  here. 

When  they  begin  applying  their  new  skills,  they  should  have  a  grace 
period  in  which  tolerance  for  error  is  higher  than  normal. 

Related  techniques 

Limit  Introduction 
Learn  By  Doing  Only 
Learn  By  Mucking  About 
Post-Training  and  Transfer 
Feedback— Action  Vs  Learning 

References 
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VISUAL  CUES  AS  PROMPTS 

Description 

If  you  are  training  someone  to  do  something  that  you  can  show  them 
how  to  do  in  detail,  and  the  task  requires  choices  to  be  made  based 
on  what  they  see,  it  can  be  a  good  idea  to  supplement  the  normal 
visual  cues  with  strong  visual  cues  (e.g.,  color  cues)  that  are  easier 
to  notice  and  understand.  This  can  be  looked  at  as  a  form  of 
prompting. 

How  to  do  it 

Add  visual  cues  that  indicate  different  actions,  e.g.  red  means  turn 
the  handle  left,  green  means  right.  Make  the  cues  part  of  the 
background,  trying  not  to  overshadow  the  actual  cues  that  you  are 
training  the  person  to  use,  but  accentuating  those  cues. 

One  thing  to  note  is  that  if  you  put  in  artificial  visual  cues,  the  person 
may  not  learn  to  make  use  of  the  meaningful  cues  that  are  present, 
because  the  artificial  ones  are  enough.  So  you  have  to  ween  them 
from  those  fairly  quickly,  and  make  the  correct  cues  salient.  Holding 
also  suggests  that  you  use  the  color  cues  for  one  problem  at  a  time, 
and  let  them  learn  to  do  each  right  before  moving  on  to  the  next,  so 
that  they  do  not  become  too  dependent  on  the  artificial  cues. 

Indicators 

This  approach  can  be  used  when  training  tasks  based  on  visual 
perceptions  and  where  the  cues  are  not  ambiguous.  This  doesn't 
mean  they  are  easy  to  see,  necessarily,  but  in  the  presence  of  a 
certain  cue,  it  is  possible  to  decide  exactly  what  to  do  (so  that 
associating  a  certain  prompt  with  that  cue  is  always  appropriate). 

If  there  is  some  other  type  of  indicator,  e.g.,  a  sound,  or  a  touch,  that 
can  be  added  to  the  same  effect,  that  would  also  work.  The  original 
author,  Holding,  only  discussed  visual  cueing.  An  extension  to 
Holding's  work  could  be  to  use  auditory  cues  with  auditory 
discrimination  tasks,  or  touch  cues  with  touch-based  ones.  It  may 
also  be  possible  to  mix  sensory  cues,  using  for  example  an  auditory 
cue  for  a  visual  discrimination  task.  Holding's  suggestion  is  that  you 
use  the  cue  as  a  background  for  the  actual  cue  that  they  are 
supposed  to  learn  to  use. 
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Counterindicators 

Putting  in  visual  cues  along  the  way  may  be  very  difficult,  if  it 
interferes  with  the  task  process  or  interface.  You  wouldn't  want  to 
add  a  bunch  of  little  lights  to  the  console  of  a  plane,  for  example,  or 
try  to  add  artificial  cues  to  a  basketball  game  -  though  both  of  these 
are  activities  where  decisions  are  based  on  what  you  see. 

Common  problems 

Don't  allow  trainees  to  become  overdependent  on  the  artificial  cues. 

Related  techniques 

Visual  Cuing  For  Navigation 
Avoid  False  Recognition 
Feedback 

References 

Holding 
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VISUAL  CUES  FOR  NAVIGATION 

Description 

For  motor  tasks  such  as  the  laboratory  maze  of  tracking  tasks,  it  is 
good  to  show  the  subject  where  to  go,  instead  of  letting  them  use 
trial  and  error.  You  do  this  by  marking  the  correct  path  with  a  visual 
cue.  This  works  better  when  you  do  not  merely  mark  the  correct  path, 
but  also  mark  as  wrong  the  ones  that  the  subject  should  not  take. 

How  to  do  it 

At  each  decision  point  (fork  in  the  road)  put  an  indicator  telling  the 
trainee  whether  it  is  a  place  to  turn.  Also  mark  places  where  they 
should  never  arrive  so  that  they  get  back  on  the  right  track  if  they 
make  a  mistake. 

Progressively  remove  the  cues,  let  the  trainees  try  it  (using  the 
remaining  cues)  and  then  put  the  cues  back  so  that  they  can  once 
again  follow  them,  but  this  time  with  the  natural  cues  also  being 
noticed. 

Indicators 

When  training  someone  in  navigating  or  tracking.  It  only  works  when 
they  are  going  to  get  to  practice  actually  going  through  the  target 
place,  because  they  have  to  see  the  cues. 

Counterindicators 

If  you  can't  give  them  visual  cues  then  this  technique  might  not  be 
beneficial.  For  example,  it  may  not  help  to  say  "don't  turn  at  the  big 
cow  sign,"  even  though  it  would  help  if  you  could  put  a  red  marker  on 
the  cow  that  indicates  they  should  not  turn  there. 

This  technique  takes  some  effort,  especially  in  a  real-world  setting. 

Common  problems 

If  people  come  to  rely  on  the  cues  you  put  in  place,  they  will  not  pay 
as  much  attention  to  the  cues  intrinsic  to  the  path.  If  you  want  them 
to  learn  how  to  navigate  without  the  cues,  then  you  have  to  make 
sure  they  notice  the  intrinsic  cues. 
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Related  techniques 

Visual  Cues  As  Prompts 
Avoid  False  Recognition 
Learn  By  Doing  Only  (Alternate) 

V 

References 
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